RAW Workflow

[Hans Nystroem]

Any thoughts about it?
To my knowledge there is no NLE that can work with RAW-files?
(perhaps a Quantel or some other very exotic turnkey NLE, but...)

Or do someone have information within "the industry" about something "round the corner"?

Regards
Hans

[turing]

tif sequences, quicktimes containing uncompressed fullres frames, cineon sequences and DPX files are all fairly easy to work with in final cut pro.

the stuff I've done testing 2k usually involves making a proxy to DVCPRO HD for editing, then ripping out a 2k version when I'm done by changing the sequence presets. obviously you have to watch your timebase.

anyway, works great.

_a

[GabrielR]

I assume you mean RAW format files pre-color conversion?

A friend worked on a indy shot on the Viper in RAW. He said they edited a proxy then did the final color conversion on the raw files in Luster.

Not sure how they did the conform. But they were editing on a high end Avid solution.

[John Allardice]

Remember also that you wont need a DVCPRO HD proxy for editing, you'll be editing in REDCODE (RAW) on your NLE of choice.
At least, I'm assuming that this is what Graham's contributions are intended to lead to.

J

[taubkin]

Redcode is not raw... raw is raw. Redcode is compressed, isn't it? Is there anything I'm missing?

[Stuart English]

I think that the definition of RAW is getting confused between its use in RED and stills cameras and its use in Viper.

RED's RAW means - 12 bit per pixel Bayer pattern data off the CMOS sensor, (similar definition to a stills camera) unprocessed in terms of gamma, white balance etc etc

Viper's RAW seems to mean RGB data off the CCD sensors, unprocessed in terms of white balance, but mapped to 10 bit log 1920 x 1080 progressive.
IMHO a less confusing name for this RGB mode is "FilmStream". RED has an equivalent user option for 2K resolution RGB recording and HD-SDI output.

Compression is a separate discussion. There is nothing inherently different about the Viper FilmStream RGB signal after it has been recorded to an HDCAM SR than when it comes out of the camera over dual link HD-SDI, except for any compression artifacts the recording itself introduces.

Same for RED or stills camera RAW. Until this point in time, all RAW stills have been captured uncompressed, but there is nothing in theory that says you couldn't also compress Bayer RAW - it would be unconventional for sure, but possible.

As REDCODE is under development I won't comment any further on that specifically, there are several posts on that subject on this board and in other forums.

[Damon Botsford]

Remember also that you wont need a DVCPRO HD proxy for editing, you'll be editing in REDCODE (RAW) on your NLE of choice.
At least, I'm assuming that this is what Graham's contributions are intended to lead to.

J

I could be mistaken, but I thought I read that you could output REDCODE (for proxy) and Raw simultaneously from the camera, although I'm not really sure why you would want to do that. Just stick with REDCODE and avoid bookoo bucks working with RAW. I have a feeling it's gonna be an incredible codec. 10 bit 4:4:4 2K... Sweeeeeeeeet!
I'll worry about RAW later... much later.

[Thomas Mathai]

It makes sense to edit proxies and then conform the high rez to the edit. I would rather have FCP or other NLEs have proxy capability.

After Effects does this nicely. You can load a 2k sequence, then in the project window, point to a lower resolution version of the shot as the proxy. When it comes time to render, just make sure proxies are turned off.

I would prefer either RED generate proxies automatically when shooting RAW, or FCP give the option to generate proxies automatically in a proxy mode.

[Damon Botsford]

I have to agree with you there. With ever increasing data rates, every NLE should have the capability to edit proxies and conform later. It would be a nice feature to have down the road. I'm just hoping REDCODE makes this obsolete. Of course, I thought DVCPRO-HD was the end all. Not quite. No need for me to hijack the thread... back to RAW

[Thomas Mathai]

I could be mistaken, but I thought I read that you could output REDCODE (for proxy) and Raw simultaneously from the camera, although I'm not really sure why you would want to do that. Just stick with REDCODE and avoid bookoo bucks working with RAW. I have a feeling it's gonna be an incredible codec. 10 bit 4:4:4 2K... Sweeeeeeeeet!
I'll worry about RAW later... much later.


Compressed codecs are fine if computing power and storage limitations are a big concern.

A big film can afford to deal with those issues easier than indies, so it would make more sense to shoot RAW.

RED is aiming for the full range of users.

[Hans Nystroem]

Ted Schilowitz is a member of the RED-team...
Would it be possible to accelerate RAW with a dedicated card?
I am thinking collaboration between RED and AJA now...

[Hans Nystroem]

Clarification of above...

"Same for RED or stills camera RAW. Until this point in time, all RAW stills have been captured uncompressed, but there is nothing in theory that says you couldn't also compress Bayer RAW - it would be unconventional for sure, but possible." -Stuart English

So I was thinking of "compressed" RAW.

[Jarred Land]

Ted Schilowitz is a member of the RED-team...
Would it be possible to accelerate RAW with a dedicated card?
I am thinking collaboration between RED and AJA now...

as far as I know Ted still has a very good relationship with AJA so its very possible, and a hardware card really makes sense. Probally would be more like a collaboration between RED, AJA with a little help from a NLE maker like Apple.

[Hans Nystroem]

If they could engage Apple in the process it would be great!
Apple have "Aperture", so they have interest and knowledge about RAW.
Perhaps FCP-X (extreme or number) could handle RAW natively.
And with the help of a dedicated hardware card make the workflow fly!

A bit strange that Apple didn´t show anything at NAB...hm.

[therock]

Nice idea, but until now AJA makes no processing on the their cards except scaling. Thats when the card "supports" codecs. Unfortunately no coding/decoding on the board yet. Don't want to be negative, but that card would cost a fortune. Think about WMV accelerators and such. But I would appreciate that too.

Editing in true/raw whatever native of 4k or so is not necessary. Nobody wants to shift around the big files/sequences. Think about editing on a laptop and all the big things become unrealistic or uninteresting completely. Except HD, nobody is currently
editing in 2k or 4k. Not even one the 'High-End' systems as they are simply too expensive to fuddle around for 3 months on a film-edit. A high-quality proxy-mov would be nice. After picture-lock press conform, go home and tomorrow its done.

On the other hand, if you talk about image sequences there are some very special truths in picture handling and quality in the Apple-Post world.

- Quicktime and FCP support image sequences very bad.
In my experience FCP simply cannot process them. Image sequences are imported as individuals. Tied to OSX and Quicktime only for image sequence support, your practically limited to a very low number of images from a sequence.
Ever tried to make a QT out of a 15000 frames image sequence ? Good night and good luck. And thats only 10 minutes at 25 or 8.x in 30 fps. And this non-support is official. Its even written in the help pages.

- Quicktime incorporates a gamma-shift when you move from image sequences.
Compositing becomes a horror when mixing Quicktimes and image sequences as the Quicktimes make that gamma shift regardless of their origin and impossible to switch it off.

- Quicktime doesn't support log-files.
You're tied to a plug-in from a programmer somewhere out there. But thats only a way, unfortunately not a solution, when you think about sequence support in general (andw ant to do DI/Film)

My summary of film support based on image sequences: poor.

Solution might be indeed a simultaneous support of native (raw, red-code..) and a proxy version of it at the same time.
One advance of image sequences over Quicktime:
Ever got a corrupt Qucktime ? Or recording broke before finishing. Or very stupid: power loss during capture ? That Quicktime is corrupt. And probably stays so. An image sequence is complete except the last (broken) frame.
Think about your shoot, a nice unrepeatable session and the 60 min quicktime gets corrupt right at the end.


I can think of 2 ways of how people want to work:

- Image sequence based work for high end post, feature film, highest quality, safety blabla. Problem: You need videos from the sequences for review and editing.
Question: How to get an according Quicktime fast from a image sequence without ages of converting.

- complete QT work style for indies, docs etc. Problem: They cannot afford full-res editing or mobile resources (Laptop-Editing etc.) limit that possiblity.
Question: A small QT as an exact copy from the original again with less or no waiting.


Suggested solution: both ways would benefit from a simultaneous recording of two streams.

Version 1:
Image sequence capture and a Lower-Res QT in parallel. Editing in FCP then High-End post.

Version 2:
Native recording QT (Red-Codec) and Lower-Res in parallel for those finishing projects completely in FCP.

RED: possible ? (Without codex; )

[stokestack]

"- Quicktime incorporates a gamma-shift when you move from image sequences.
Compositing becomes a horror when mixing Quicktimes and image sequences as the Quicktimes make that gamma shift regardless of their origin and impossible to switch it off."

That depends on the app. There are lots of gamma concerns with QT.

If you must have QTs from sequences, your best bet (as far as I know) is Shake. Not only was it designed primarily to handle image sequences, but it suppresses QT's gamma-fiddling for exactly the reason you state: You never want these adjustments to wind up in your composites.

If you want lower-data-rate proxies, I'd suggest the DV-based codecs, 50 or 100. If you want full res but minimal bloat from image sequences, check out BitJazz's Sheer codecs, which use lossless compression and don't screw around with the gamma.

[Graeme_Nattress]

Yes, Shake does suppress the Gamma daftness of QT. But FCP does not, and that's something we've all got to bug Apple about to fix (speaking as Graeme personally here, not as RED). I mean, it's just not "pro". I think AE might also avoid the gamma weirdness for the most part too, and depending upon how a QT codec is written it might be avoidable in FCP, but you'd still have levels issues on stills.

Graeme

[spamagnet]

Is Red going to document the raw format? It would be nice to convert it to any useful format, such as OpenEXR. Actually, I'm surprised I haven't seen much discussion about OpenEXR. It supports the "windowing" feature of the Red camera and seems like an excellent format for FX-heavy footage.

[Graeme_Nattress]

We'll make sure you can get access to the raw data to feed into your own conversion app, should you need to convert it to a file format we don't support.

Graeme

[spamagnet]

We'll make sure you can get access to the raw data to feed into your own conversion appSounds good. Thanks for the quick reply!

[therock]

...I think AE might also avoid the gamma weirdness for the most part too, and depending upon how a QT codec is written it might be avoidable in FCP, but you'd still have levels issues on stills....Graeme

AFAIK the gamma shift comes in place in the moment you use the QT interface, so when you render and your app connects to to QT you get the shift. Indeed shake seems to surpass that, as it may have its own interface.


...It would be nice to convert it to any useful format, such as OpenEXR.

dpx and EXR would be cool for post. dpx is more widely used, while exr is more versatile, but support is rare. Both formats support metadata, which is necessary for further steps. an interesting question is how fast the internal dsp really is. EXR supports lossless compression and the data could be compressed before writing.
But find converters and editors for EXR. Don't think about FCP here.
oops.

[Graeme_Nattress]

It all depends upon how the app asks for the frames from quicktime. You can get around the gamma nonsense by writing your codec to give native gamma when asked for a different gamma, or in an app, by only ever asking for native gamma, at least that's what I've been told....

Yes, FCP needs nice still image support. As much as I like FCP, that's one failing in my mind that really lets the package down.

Graeme

[therock]

Yeah, that really lets it down and its awful standards switching. Source clip, project, sequence .. boahrrr. But the features are great.

Considering the QT: unfortunately I am not a programmer. But I think the shift happens at the point of data aquisition. E.g. importing or capturing. And then its in. If common processing tools could avoid it, they would ignore the shift or better asking for the images without shift,not ? But the pictures have the shift in it, even when exported from QT or FCP... so far my experience. Had lots of trouble during production when integrating 3D and comps into a sequence. Luckily all the stuff got into cc afterwards.

[Graeme_Nattress]

That's how it looks, but that's not what happens, else how can shake avoid the gamma shift?

Graeme

[stokestack]

Two perpetrators are involved in the shift, and that's just the intentional shift; there are also problems with different gamma curves between different codecs, even when they're supposed to be applying the same one.

QuickTime knows which platform it's playing on: Mac or Windows. It makes assumptions about gamma based on the typical characteristics of those platforms' hardware. Left to its own devices, it will "correct" gamma for the playback platform. The problem here is the assumption that this is all people are doing with video: sending it right to the monitor. Back when QT was created, this was probably true. Wow, 120x90 "video" at 15 FPS on my computer! Look, Mom! But today those "corrections" get built into downstream compositions after you pull the frames from QuickTime, and that's bad.

Some codecs prevent problems by simply not performing the gamma adjustments that QuickTime requests. I think Microcosm and Sheer fall into this category.

The problem doesn't occur at acquisition, unless you count simply the tagging of the file as being from a certain source. It's impossible to know exactly why application developers don't countermand the "correction" when reading the files, but they may not even be aware of it. QT documentation isn't exactly straightforward. QT is also relatively new to the "pro" community, and these knowledgable users are calling out problems that go unnoticed by people downloading funny cat videos.

Applications that are wise to QT's shenanigans can try to prevent them by requesting "source" gamma whenever they call QT to decode a file. This is what Shake does, essentially saying "Don't mess with it" when reading files. When writing QT, Shake specifies standard video gamma, since it has to pick something.

Still images are troublesome for QT because they aren't tagged with any gamma, so QT doesn't know what the source is. So it starts making assumptions. I have to admit, I don't remember what the assumption is for stills. But frankly, QT is not what you want to use for stills. Historically, it has reduced everything down to 8-bit color; I think that might finally have been addressed for a couple of formats, but I'm not sure.

Another problem to be aware of is dynamic-range truncation when going from YUV to RGB in QuickTime. Anyone who needs to use QT should be very glad that the Red will be an RGB device, because currently there's no way to retain superwhite/superblack data when using QT to go from YUV to RGB with most codecs (I think Sheer has one that will do it). So instead of 1-255 (deepest black to whitest white), your RGB contains only what was between 16-235. Make no mistake, this is a noticeable loss.

[Graeme_Nattress]

Very good sumarry of the issues, Stokestack!

Graeme

[therock]

yes, thanks for that summary!

As suggestion for a solution: why not taking QT as frame-container ?
That way it could handle image sequences, you might not suffer from the fact that an unfinished QT is corrupt (as with a written stream). And you could use the format of the image, not necessarily an imbedded codec. I think QT handles tiff and tga sequences this way.

- Sounds weired. Just looking for a way to get OSX and QT handle image sequences.

[stokestack]

Hm, Rock, not quite sure what you mean. You mean using a QuickTime file as a simple wrapper around raw data? You could, but you just have to have some codec component that will know how to get the data out. If you want QuickTime to manipulate such a file, you have to write a valid component that will unpack that data into some kind of pixel format that QT understands.

This would hardly seem worth the effort, given that there are QT codecs on the market that retain fidelity AND offer some lossless compression to boot. I'd say if you really want to pack image sequences up in QTs, buy Shake and the Sheer codecs and knock yourself out. Lucky for you, Apple just knocked $2000 off Shake.

Failure to support image sequences isn't really a QT problem. It's a problem of applications that are so dependent on QuickTime that they can't handle anything else. Blame the application developers, who took the quick and dirty path instead of writing a format-agnostic architecture that is extensible and forward-thinking.

[toke lahti]

Apple's color management is really a mess.
You propably all know this:
http://manuals.info.apple.com/en/Shake_4.1_lbn_z.pdf
"Differences in Gamma Handling Between Shake and Final Cut Pro"

So qt assumes that all rgb material has gamma of 1.8 and all YCbCr material is 2.2 and does these secret, undocumented (in fcp's manual) and mandatory gamma conversions.
Their solution to use display gamma of 1.8 makes harder to use photoshop and after effects for hdtv with gamma of 2.2.
I hope fcs6 will have some real color management and it will arrive soon!

[toke lahti]

...buy Shake and the Sheer codecs and knock yourself out.
Problem is that you cannot edit with shake.

[Jarred Land]

yeah but you can do conversions with shake.. not very romantic i know but it works.

[meta-lingo]

I deal with 4K RAW sequences a lot as I shoot tons of timelapse with a 14 Mpixel SLR camera. It is a bitch. Thankfully, After Effects 7 supports RAW image sequences which has eliminated the need for batch processing work-arounds. It is still incredibly slow on my Quad-Core G5, but the image quality is stunning. This is the film killer format.

Seems like the thing to do is to have the camera/recording system create the proxy while shooting--like a making a simultanious work print and negative. The SLR does this by writing a low res JPG at the same time as the full res RAW and it is a handy feature.

[therock]

..You mean using a QuickTime file as a simple wrapper around raw data?

Yep, thats exactly it.


..You could, but you just have to have some codec component ... you have to write a valid component that will unpack that data into some kind of pixel format that QT understands.

Unfortunately I know.


..This would hardly seem worth the effort, given that there are QT codecs on the market that retain fidelity AND offer some lossless compression to boot.

My idea was to simplify the workflow.
OSX and FCP cannot handle image sequences proper. Sometimes I wonder how shake does it. I assume due to its own interface. But when you open folders using system dialogue, like QT does, you get mad. No way for big long images sequences.
Hence my idea to JUST-WRAP them.
Another codec needs to encode, even if its fast. If its a wrapper, native format is in.

I just try to eliminate the gap between QT and image sequence as in a professional environment you often need both.


..Failure to support image sequences...It's a problem of applications that are so dependent on QuickTime that they can't handle anything else. Blame the application developers, who took the quick and dirty path instead of writing a format-agnostic architecture that is extensible and forward-thinking.

Yeah maybe, but again: I think its due to Apples interface of system dialogue.
Just open up a folder with 30.000 images on a Mac. UNUSABLE.
And every app thats connects to the file dialogue of OSX happens to have the same problem
- Even Quicktime!

(Again: Shake obviously not, as it has its own interface, connecting directly, taking advantage of the linux subsystem, not having to wait for system response.)

[therock]

I deal with 4K RAW a lot .. It is a bitch.. It is still incredibly slow on my Quad-Core G5..

Seems like the thing to do is to have the camera/recording system create the proxy while shooting--like a making a simultanious work print and negative. The SLR does this by writing a low res JPG at the same time as the full res RAW and it is a handy feature.

Exactly my wish. Same with QTs.
When you should have QTs with RED-CODEC, it would be perfect to not have to render proxy or "off-line" QTs.
For high-end uses it would be RAW and maybe JPGs in a little smaller size.
(half or third)
- but there you go: Writing to streams at the same time, and one being a copy from the first, wheeeew. Lots of processing to to, and writing to disk still being unsolved.

[Jarred Land]

I like the proxy idea with the Raw files.. as long as there was software to really link them. It would be perfect if you could either do a proxy in or off camera, and use the proxy for cutting and and a couple raw frames for color timing.. and one package integrated the output into whatever deliverable you needed from the RAW file with edl and color timing information.