F55: YUV or ProRes darken my slog2 footage?

starcentral

starcentral

Veteran
Hi guys,

I want to create some high quality uncompressed 10-bit files from my 4k 10-bit XAVC MXF files to give to a digital FX artist for some work.

I'm trying to figure out the best output setting from Davinci but if I use ProRes or QT Uncompressed 10-bit YUV the footage gets a tad darker I notice, almost like some contrast is added.

Any tips here?
 
Probably best to go uncompressed DPX sequence. Often, FX work goes faster with single frames since they can be individually manipulated and saved without re-saving the entire shot.

Cheers.
 
Yeah the DPX output looks good and seems to preserve the slog2 flat look, I was hoping for just a single file though not a file sequence. I tried QT uncompressed 10-bit RGB but my After Effects doesn't open it. Then I tried QT uncompressed 8-bit RGB and my AE opens it but it's only 8-bit and I still need to grade it later.

I will probably stick with DPX 10-bit.
 
Dennis,

You can also export OpenEXR files if you turn on ACES.

The method is:

-Turn on ACES

-Set IDT to Sony SLog2 or Sony raw

-Leave ODT as "No Transform"

-Render to "EXR". Set type "RGB Half"

This makes a bunch of OpenEXR files that are very similar to DPX, but the Slog2 or raw is linearized perfectly and stored in a 16bit linear DPX-like container (an awful lot like Sony raw, except on compressed). Means all latitude is saved, no ifs ands or buts, and handed to FX guy in the most pristine way possible.

FX guys like to work with linear light files because one, they don't have to worry about LUTs for inputing footage, and two, it takes the look of any LUTs out of the picture, so to speak. Lots of compositing operations work better in linear space as well, Stu Maschwitz years ago did a lot of writing about how this is preferred.

Files are huge, but that what you get when you want quality. :-)
 
Not sure with Davinci, but make sure when exporting you turn off baked in gamma correction ( i think ? ) with Pro Res . This is a known bug using QuickTime codecs and seems to appear randomly.
 
Thanks Nate I will give that a try.

The idea is I will be doing the final grade in Davinci using 1.) the XML file exported from NLE, 2.) any effect shots that are returned to me from the fx artist.

Once all these new clips are rendered out of Davinci they can be relinked to the original edit and project rendered out to final output.

If OpenEXR files become linearized, would I be able to use the same grade I come up with for the slog2 version files, or will I have to treat/grade the effect clips (openEXR) differently.


Surf It Mike, yes I recall an issue with QT because Apple was handling gamma somehow differently. This is ok if all the original information is still there but I don't recall the deal with this, or if there was a fix.
 
Last edited:
starcentral said:
Thanks Nate I will give that a try.
Click to expand...

If you do, you can check the .exr files by opening in Photoshop. They will open as 32bit float files, and you can bend the exposure slider to see that it's all there (but PS won't show the entire latitude in window all at once)

starcentral said:
The idea is I will be doing the final grade in Davinci using 1.) the XML file exported from NLE, 2.) any effect shots that are returned to me from the fx artist.

Once all these new clips are rendered out of Davinci they can be relinked to the original edit and project rendered out to final output.

If OpenEXR files become linearized, would I be able to use the same grade come up with for the slog2 version files, or will I have to treat/grade the effect clips (openEXR) differently.
Click to expand...

It depends on how the FX guys want to export the files. They are probably going to be keen to give you Cineon-log DPXs, which there are LUTs for in Resolve. I imagine they will need small tweaks on the grade, yes.

If they give you OpenEXRs back, I don't know if you need LUTs for those or you need to be working in ACES.


starcentral said:
yes I recall an issue with QT because Apple was handling gamma somehow differently. This is ok if all the original information is still there but I don't recall the deal with this, or if there was a fix.
Click to expand...

The big QT gamma debacle. My take on it has always been, each program uses different drawing methods to render QT onscreen. QT Player 7 would use one gamma, FCP7 would use another, AE yet another. Add in YUV to RGB conversions that each program may or may not be performing, and it's the wild west.

I have never had problems judging my files by a proper AJA or Blackmagic card. They always, since 2005, seemed right on to me. Takes the OS X/screen gamma stuff right out of the equation.
 
Lenilenapi said:
What is ACES?
Click to expand...

Here's an overview and explanation taken from this post written some time ago: http://www.dvxuser.com/V6/showthrea...EC-709&p=1986295141&viewfull=1#post1986295141

starcentral said:
The concept of ACES was invented in 2004 by the Academy of Motion Picture Arts & Sciences and they say they have been undergoing industry-wide production trials with the major Hollywood studios.

ACES is the concept that cameras work in a common color space, or conform to a "manufacturer neutral" color space. ACES is a color space that is the absolute widest possible that most cameras or even film can not fill. That's how big and largely defined ACES color space is - its even larger than the visible spectrum! (that's right - colors you can't even see with your eye!)

By working in a "standard" or "common" color space (in this case ACES) it means a more common interchange could exist so that you could intercut, mix and match, or composite from various acquisition devices, etc.. without any color descrepencies in the workflow process. ie. blue becomes the color same color "blue" no matter what camera it came from or where it is in the workflow process or who's working on it.

But will other manufacturers be reluctant to adapt or conform to ACES? Because very unique image characteristics are determined by a camera's specific sensor or even the dyes used in the CFA (color filter array) - many manufacturers develop their own "color science" (or special tricks) to perfect the overall image and do away with noise or other imperfections when certain colors or hues are seen by the camera. It is why all cameras are known for their "look" as people call it. ACES in practical terms strips away some of that "secret sauce" because the camera must conform to the ACES standard color space and not its own. So if a camera just can't handle a sepecific shade of violet/purple and breaks apart into noise then too bad, in ACES it will show through. For that reason I can see why ACES might be slow to adapt. Imagine what RED would think of with someone elses color science applied to it?

So what Sony is saying with the F55 and F65 is that if you shoot in the raw format you can conform the bayer data directly into the ACES standard (color space).

From a technical perspective I don't see how this will benefit the image today or make it better because I think of ACES as one giant LUT that just conforms the image from the F55/F65 to meet a certain predefined specification. It "may" or "may not" happen to look good based on what kind of sensor or image issues come out of removing any of the "special sauce" the camera manufacturer wanted there in the first place.


Recommended reading on ACES:
http://www.fxguide.com/featured/the-art-of-digital-color/
http://www.oscars.org/science-technology/council/projects/aces.html
Click to expand...


I don't know where ACES currently stands in the industry or who's using it. They've been trying to get it off the ground for almost a decade and one flaw I see with it is that even once you profile various camera manufacturers, ACES can never account for what glass was actually put on a camera and wouldn't all the combinations of various glass throw such a spec profile off? Ah, what do I know.
 
Nate Weaver said:
The big QT gamma debacle. My take on it has always been, each program uses different drawing methods to render QT onscreen. QT Player 7 would use one gamma, FCP7 would use another, AE yet another. Add in YUV to RGB conversions that each program may or may not be performing, and it's the wild west.

I have never had problems judging my files by a proper AJA or Blackmagic card. They always, since 2005, seemed right on to me. Takes the OS X/screen gamma stuff right out of the equation.
Click to expand...

Most of what I read online about the gamma debacle is where people experienced brighter videos on their playback in QT based on their original.

My issue is that when I create a ProRes file from Davinci using my slog+sgamut MXF file some contrast is added, and it looks like some of that slog2 wash effect is removed. It actually looks like what would happen if you chose 64-940 as your levels instead of 0-1024.

For the FX I can definitely create the EXR or DPX files, but I'm eventually doing a round-trip of files graded in Davinci to replace the edit files and would like to gaurantee myself I'm not going to have problems with the contrast, brightness or saturation shifting around on me.
 
Ok so here is original 4k XAVC 10-bit 422 slog+sgamut file loaded up in Davinci then outputed to three different formats; DPX, EXR, & ProRes.

Note the histogram and where the gamma lines up relative to the actual video signal:

DPX (looks exactly identical to the slog2 file as it appears in Davinci or "in-camera" on playback)
dpx_zpse0908d33.jpg


ProRes (shifts the gamma making image darker)
prores_zps71ac93cf.jpg


OpenEXR (also shifts the gamma. this is linearized light)
exr_zps43210fe9.jpg
 
starcentral said:
It actually looks like what would happen if you chose 64-940 as your levels instead of 0-1024.
Click to expand...

BTW, that could be exactly what's going on. There's file interpretation options if you right click on the media in the AE media list. Video vs. data.

starcentral said:
For the FX I can definitely create the EXR or DPX files, but I'm eventually doing a round-trip of files graded in Davinci to replace the edit files and would like to gaurantee myself I'm not going to have problems with the contrast, brightness or saturation shifting around on me.
Click to expand...

I don't think there can be a guarantee, especially if you are looking at your computer screen instead of an SDI out. I think the best you can hope for is not clipping data and keeping a handle on Log conversion issues. Besides, it's going to be conformed into your Resolve timeline where you can tweak inconsistencies anyway? Who's to say your FX guy isn't going to mess with contrast and gamma a little to hide masks and his work. I've seen artifacts of effects work brought out with gamma changes on projects I've been involved with.

OpenEXR side-steps all that because it's more of a linear data container rather than something that is designed to be looked at on a screen and deemed to be correct or incorrect. There is no gamma in it to screw up. It's all in how the program reading it is set up to display it, just like raw files.

It's true I'm guessing based on what I've picked up from my FX friends and what I've read. I'd find an Effects Supervisor if you really want the best advice?
 
Ok so I take the darkened QT prores 422 HQ output and add 'levels' filter to it in AE and adjust the input black from 0 to -20 and input white from 255 to 275 and it looks identical to the "slog2" DPX version in terms of contrast, brightness and gamma.

I guess that's sort of a relief but not really.

*Edit: Maybe, just maybe its the brighter version DPX that is the issue? I will try a version out with 64-940 selected and see how it looks compared to the ProRes one, I bet they will be identical.

So damn it which one is "correct"?
 
Progress?

Progress?

Hey Nate (or anyone) can you confirm something on the Davinci RGB parade when dropping a slog2 + sgamut clip into Davinci and setting the clip attribute data levels to 0-1023 in the media pool?

Here's what a typical waveform looks like when I use those settings:
ScreenShot2013-06-23at30925PM_zpsa11f0475.png


Notice the blacks are jacked up near 64'ish. I never thought anything of this before because I assumed the washed out look was what slog2 looked like. But now I'm starting to think otherwise.

When I set the clip attributes to "auto" or 64-960 I get something like this which coincidentally is pretty exact to how the ProRes video files have been coming out even when I leave all settings in Davinci to 0-1023.

ScreenShot2013-06-23at32406PM_zps8095511e.png



Now here what's muddy. If applying any of the Sony LUTs it looks way better on the 0-1023 version and pulls the blacks down from the 64 range to zero. If you apply Sony LUT on 64-960 version you clip the blacks and end up with way too much contrast in your image. That of course can be fixed by adding a node before the LUT and using the LIFT and GAMMA wheels to bring things back to normal.

What an oddity all this.
 
Ok I no longer believe this is an issue and that everything is actually working by design.

It is not a software issue or problem with Apple or ProRes, Davinci, etc.. Basically I had set my Davinci to work with "data levels" which is the 0-1023 range but certain codecs & formats no matter how you set your "output levels" will only be 64-960 (legal). That was my issue. I was expecting a full range out of a format that can't be full range.

Page 168/169 of the Davinci manual does a really great job at explaining that only SOME FORMATS can store the full data level range ie. 0-1023.

DPX image sequence, or ProRes 4444 as QT time are two of them.

It all makes sense now.
 
You must log in or register to reply here.
Back
Top