RAW Workflow

Graeme_Nattress said:
...I think AE might also avoid the gamma weirdness for the most part too, and depending upon how a QT codec is written it might be avoidable in FCP, but you'd still have levels issues on stills....Graeme
Click to expand...

AFAIK the gamma shift comes in place in the moment you use the QT interface, so when you render and your app connects to to QT you get the shift. Indeed shake seems to surpass that, as it may have its own interface.

spamagnet said:
...It would be nice to convert it to any useful format, such as OpenEXR.
Click to expand...

dpx and EXR would be cool for post. dpx is more widely used, while exr is more versatile, but support is rare. Both formats support metadata, which is necessary for further steps. an interesting question is how fast the internal dsp really is. EXR supports lossless compression and the data could be compressed before writing.
But find converters and editors for EXR. Don't think about FCP here.
oops.
 
It all depends upon how the app asks for the frames from quicktime. You can get around the gamma nonsense by writing your codec to give native gamma when asked for a different gamma, or in an app, by only ever asking for native gamma, at least that's what I've been told....

Yes, FCP needs nice still image support. As much as I like FCP, that's one failing in my mind that really lets the package down.

Graeme
 
Yeah, that really lets it down and its awful standards switching. Source clip, project, sequence .. boahrrr. But the features are great.

Considering the QT: unfortunately I am not a programmer. But I think the shift happens at the point of data aquisition. E.g. importing or capturing. And then its in. If common processing tools could avoid it, they would ignore the shift or better asking for the images without shift,not ? But the pictures have the shift in it, even when exported from QT or FCP... so far my experience. Had lots of trouble during production when integrating 3D and comps into a sequence. Luckily all the stuff got into cc afterwards.
 
Two perpetrators are involved in the shift, and that's just the intentional shift; there are also problems with different gamma curves between different codecs, even when they're supposed to be applying the same one.

QuickTime knows which platform it's playing on: Mac or Windows. It makes assumptions about gamma based on the typical characteristics of those platforms' hardware. Left to its own devices, it will "correct" gamma for the playback platform. The problem here is the assumption that this is all people are doing with video: sending it right to the monitor. Back when QT was created, this was probably true. Wow, 120x90 "video" at 15 FPS on my computer! Look, Mom! But today those "corrections" get built into downstream compositions after you pull the frames from QuickTime, and that's bad.

Some codecs prevent problems by simply not performing the gamma adjustments that QuickTime requests. I think Microcosm and Sheer fall into this category.

The problem doesn't occur at acquisition, unless you count simply the tagging of the file as being from a certain source. It's impossible to know exactly why application developers don't countermand the "correction" when reading the files, but they may not even be aware of it. QT documentation isn't exactly straightforward. QT is also relatively new to the "pro" community, and these knowledgable users are calling out problems that go unnoticed by people downloading funny cat videos.

Applications that are wise to QT's shenanigans can try to prevent them by requesting "source" gamma whenever they call QT to decode a file. This is what Shake does, essentially saying "Don't mess with it" when reading files. When writing QT, Shake specifies standard video gamma, since it has to pick something.

Still images are troublesome for QT because they aren't tagged with any gamma, so QT doesn't know what the source is. So it starts making assumptions. I have to admit, I don't remember what the assumption is for stills. But frankly, QT is not what you want to use for stills. Historically, it has reduced everything down to 8-bit color; I think that might finally have been addressed for a couple of formats, but I'm not sure.

Another problem to be aware of is dynamic-range truncation when going from YUV to RGB in QuickTime. Anyone who needs to use QT should be very glad that the Red will be an RGB device, because currently there's no way to retain superwhite/superblack data when using QT to go from YUV to RGB with most codecs (I think Sheer has one that will do it). So instead of 1-255 (deepest black to whitest white), your RGB contains only what was between 16-235. Make no mistake, this is a noticeable loss.
 
Last edited:
yes, thanks for that summary!

As suggestion for a solution: why not taking QT as frame-container ?
That way it could handle image sequences, you might not suffer from the fact that an unfinished QT is corrupt (as with a written stream). And you could use the format of the image, not necessarily an imbedded codec. I think QT handles tiff and tga sequences this way.

- Sounds weired. Just looking for a way to get OSX and QT handle image sequences.
 
Hm, Rock, not quite sure what you mean. You mean using a QuickTime file as a simple wrapper around raw data? You could, but you just have to have some codec component that will know how to get the data out. If you want QuickTime to manipulate such a file, you have to write a valid component that will unpack that data into some kind of pixel format that QT understands.

This would hardly seem worth the effort, given that there are QT codecs on the market that retain fidelity AND offer some lossless compression to boot. I'd say if you really want to pack image sequences up in QTs, buy Shake and the Sheer codecs and knock yourself out. Lucky for you, Apple just knocked $2000 off Shake.

Failure to support image sequences isn't really a QT problem. It's a problem of applications that are so dependent on QuickTime that they can't handle anything else. Blame the application developers, who took the quick and dirty path instead of writing a format-agnostic architecture that is extensible and forward-thinking.
 
Last edited:
Apple's color management is really a mess.
You propably all know this:
http://manuals.info.apple.com/en/Shake_4.1_lbn_z.pdf
"Differences in Gamma Handling Between Shake and Final Cut Pro"

So qt assumes that all rgb material has gamma of 1.8 and all YCbCr material is 2.2 and does these secret, undocumented (in fcp's manual) and mandatory gamma conversions.
Their solution to use display gamma of 1.8 makes harder to use photoshop and after effects for hdtv with gamma of 2.2.
I hope fcs6 will have some real color management and it will arrive soon!
 
I deal with 4K RAW sequences a lot as I shoot tons of timelapse with a 14 Mpixel SLR camera. It is a *****. Thankfully, After Effects 7 supports RAW image sequences which has eliminated the need for batch processing work-arounds. It is still incredibly slow on my Quad-Core G5, but the image quality is stunning. This is the film killer format.

Seems like the thing to do is to have the camera/recording system create the proxy while shooting--like a making a simultanious work print and negative. The SLR does this by writing a low res JPG at the same time as the full res RAW and it is a handy feature.
 
Last edited:
stokestack said:
..You mean using a QuickTime file as a simple wrapper around raw data?
Click to expand...

Yep, thats exactly it.

stokestack said:
..You could, but you just have to have some codec component ... you have to write a valid component that will unpack that data into some kind of pixel format that QT understands.
Click to expand...

Unfortunately I know.

stokestack said:
..This would hardly seem worth the effort, given that there are QT codecs on the market that retain fidelity AND offer some lossless compression to boot.
Click to expand...

My idea was to simplify the workflow.
OSX and FCP cannot handle image sequences proper. Sometimes I wonder how shake does it. I assume due to its own interface. But when you open folders using system dialogue, like QT does, you get mad. No way for big long images sequences.
Hence my idea to JUST-WRAP them.
Another codec needs to encode, even if its fast. If its a wrapper, native format is in.

I just try to eliminate the gap between QT and image sequence as in a professional environment you often need both.

stokestack said:
..Failure to support image sequences...It's a problem of applications that are so dependent on QuickTime that they can't handle anything else. Blame the application developers, who took the quick and dirty path instead of writing a format-agnostic architecture that is extensible and forward-thinking.
Click to expand...

Yeah maybe, but again: I think its due to Apples interface of system dialogue.
Just open up a folder with 30.000 images on a Mac. UNUSABLE.
And every app thats connects to the file dialogue of OSX happens to have the same problem
- Even Quicktime!

(Again: Shake obviously not, as it has its own interface, connecting directly, taking advantage of the linux subsystem, not having to wait for system response.)
 
meta-lingo said:
I deal with 4K RAW a lot .. It is a *****.. It is still incredibly slow on my Quad-Core G5..

Seems like the thing to do is to have the camera/recording system create the proxy while shooting--like a making a simultanious work print and negative. The SLR does this by writing a low res JPG at the same time as the full res RAW and it is a handy feature.
Click to expand...

Exactly my wish. Same with QTs.
When you should have QTs with RED-CODEC, it would be perfect to not have to render proxy or "off-line" QTs.
For high-end uses it would be RAW and maybe JPGs in a little smaller size.
(half or third)
- but there you go: Writing to streams at the same time, and one being a copy from the first, wheeeew. Lots of processing to to, and writing to disk still being unsolved.
 
I like the proxy idea with the Raw files.. as long as there was software to really link them. It would be perfect if you could either do a proxy in or off camera, and use the proxy for cutting and and a couple raw frames for color timing.. and one package integrated the output into whatever deliverable you needed from the RAW file with edl and color timing information.
 
You must log in or register to reply here.
Back
Top