Sony's INTERNAL RAW on DaVinci Resolve

[cyvideo]

I just thought some of you may be interested.

Most of us who follow Sony's camera products and their capabilities know the following.

That the FX9, FX6 have the ability to output their RAW sensor streams. This is the stream when connected to certain Atomos recorder/monitors allows the Atomos hardware/software to convert this RAW data stream and save it as ProRes RAW.

At IBC 2024 an announcement was made that Blackmagic and Sony had reached an agreement to enable the Blackmagic Video Assist recorder/monitors to do the same. Capture and convert this sensor RAW data to Blackmagic RAW.

To do this, Sony supplied the necessary SDK to Blackmagic to allow Blackmagic to access the sensor RAW data directly off the sensors of the cameras listed above. See link below.

“Blackmagic RAW Recording From theSony FX6 and Sony FX9 - Coming Soon.”

The interesting side development to all this though is that in supplying this SDK allowing Blackmagic to access the RAW sensor data has allowed Blackmagic's Resolve, with virtually little announcement to directly access that sensor RAW data from the FX9 and FX6 from within Davinci Resolve's Color page. See BM's 18.5 announcement grab.

Since this development has taken place, owners of both the FX9 and FX6 can now process their S-LOG3 clips under Resolve's Color Page, Camera RAW tab. Under this tab you will now find a 'Decode Quality' selection named: “Full Res Sony”. The next item down the menu is 'Decode Using'. Here, you select “Clip”. Once you have selected these options, the full range of controls available under the Camera RAW tab are now available to process the RAW data contained in those S-LOG3 files. See attached grab.

For a much fuller explanation of this internal Sony RAW workflow, follow these links.

https://www.maxwellcollinsdp.com/blog/how-to-use-sony-raw-on-davinci-resolve-185

After making some inquires here in AU about what is going on here, this is what I have heard. But I cannot tell you who passed this information on, as it was strictly in confidence and off the record. It was decided that this workflow process was not in conflict with RED's no recording RAW internally patent IP. A patent that now comes under Nikon's umbrella since their purchase of RED and RED's IP.

Apparently, the argument being there is nothing in that RED patent that prevents sensor RAW data to be accessed AFTER an encoded file has been recorded and exported. As XAVC S-LOG is an encoded file, Sony are NOT recording just sensor RAW camera data within the camera in the true sense of an internal RAW recording.

I was also told to expect more cameras to be added along with further development of this new internal Sony RAW development in Resolve. Along with much more development in improving and squeezing the most out of this Resolve Sony internal RAW workflow. I've already seen improvements in the image quality since the v18.5 release up to the current version 19.1.3. Just remember if you are using this workflow any in camera noise reduction and details settings are totally bypassed. All all that takes place downstream from the RAW data stream.

I've been playing around with this workflow since 18.5 and have now actually used it on two jobs and have been totally happy with the workflow. If grading with nodes and LUTs on nodes, having the ability to process the file under the Camera RAW tab before it passes into the node tree and LUT nodes is so simple and quick. I can see this becoming a pretty standard workflow for me with S-LOG3 files coming off the cameras that are compatible.

EDIT:
I should add. The information here is in regard to the Sony/Resolve's RAW implementation when we are talking about the RAW information stored in XAVC's 10-bit log 1023 data levels files, as found in the FX6 and FX9 files. This is a world of difference to Sony's external R7 X-OCN RAW recordings, which are 16-bit linear with a 65,536 data level set. Totally different horses of a different color.

Sony/Resolve's new Sony RAW functionality adds nothing in quality improvements. It just gives you more accurate and useful control of exposure via the ISO control settings and of white balance in precise degrees Kelvin before your grading nodes. Plus control of a range of other parameters you will find under the Camera RAW tab.

Chris Young

 

[hamburger]

Nice! So it applies an inverse of the XAVC encoding to reconstruct the RGB data from the sensor? That would give better control over the WB and ISO controls compared to working with already de-bayered pictures.

I wonder if this is one of the reasons why Sony doesn't seem to be introducing a new FX model, and instead has a firmware map for the next couple years.

 

[jmone]

I've been tracking over on the Resolve forum since it first came out (as we thought it was a bug). I've attached a screen shot from one of my projects of the controls that get exposed. A couple of things I've found (at least for me),
- It only works with SLOG3/S-Gamut3.Cine (not the wider S-Gamut3) in an MXF container (from my FX6)
- you lose the ability to assign Colour Space / Gamut from the other menus for SLOG3/S-Gamut3.Cine NXF files (no big deal but why it was thought to be a bug)
- assigning other files to SLOG3/S-Gamut3.Cine does not bring up the RAW controls - eg you can't mark your A7Siii MP4 files to work with the raw tab.
- Colour Space and Gamma are always greyed out, and I can not change them (they are just for info which would makes sense).

These raw controls makes it is pretty easy to change the Colour Temp on a clip. You can select a preset, or dial it in using the colour picker on a white patch. Nice.

 

[Tom Roper]

The AXS Raw recorder for my Sony F55 snaps into a port on the back of the camera making it integral, no cables or wired external connections, and it unifies with the camera body. Because it's integral, I never thought of this an 'external' raw recorder, and apparently Red didn't either as the F55 was the first target of Red's lawsuit. Sony countersued Red for infinging on some of Sony's own patents and the parties settled. The terms of the settlement agreement are unknown but Sony's ongoing avoidance of such internal raw was voluntary by agreement, and as such Red joined the other members of the consortium informally known as 'the cartel,' Canon, Panasonic and finally Nikon among the camera makers.

So for Sony, after the F55 settlement there was never anything standing in the way of internal raw except their own voluntary compliance, which likely included not developing new external raw hardware. Enter BMD and others. But what constitutes internal raw has always been unclear at best.

To segue, Sony 16 bit linear raw is wavelet compression as was Redcode but along the line, Red switched to DCT and Sony released X-OCN.

In any event, it's been a while since I have worked with my F55 or 16b Linear Raw, but my recollection of it in Resolve was that it ran smoothly, with raw tab control on-par with BRAW. You could choose how to interpret the input as Slog2/Sgamut, Slog3/Sgamut3, Slog3/Sgamut3.cine, and possibly others. I don't remember color temperature controls working as well as BRAW (although it had them), but the other raw panel controls for black point, pivot, gamma, brightness and others were complete and fully functional in the panel. There were a few debayer options as well. Of course 4K 60 fps was the top level for the F55. BRAW 12K 72 fps on recent releases of Resolve with NVIDIA RTX4090 run smoothly.

 

[Doug Jensen]

Of course 4K 60 fps was the top level for the F55.

Tom, very good post with great info. I just need to correct on this one point. 4K RAW @ 60 fps is the limit with the F55+R5 recorder (got an R5 to sell if anyone wants one) but I switched to the R7 because it can do 4K RAW @ 120 fps on my F55. So, 120 fps is actually top level for the F55.

And yes, 16-bit RAW or X-OCN really is a joy to work with.

 

[cyvideo]

Nice! So it applies an inverse of the XAVC encoding to reconstruct the RGB data from the sensor? That would give better control over the WB and ISO controls compared to working with already de-bayered pictures.

I wonder if this is one of the reasons why Sony doesn't seem to be introducing a new FX model, and instead has a firmware map for the next couple years.

Who knows? To be honest, I see little if any difference in the output grade quality between the "Internal RAW" and LOG wheel manipulation on a calibrated monitor. The Sony RAW controls are typical of the RAW adjustments for the other RAW flavors that Resolve handles. The adjustments work well and easily reapatable. The ingested material does appear to be unaffected by NR and detail settings in the camera, so something appears to be going on under the hood. At the moment it's all guess work from there on in. The internally recorded material definitely looks better than some ProRes RAW I've seen from an FX6. CineD found the same. So currently might as well use whatever Resolve controls you feel best serves your needs.

CineD
https://www.cined.com/sony-fx6-lab-test-external-prores-raw-vs-internal-xavc-intra/

"All in all, the Sony FX6 is a solid performer in our lab test – superb rolling shutter values, and good dynamic range values are offered. The exposure latitude is about 8 stops (3 over, 5 under) – but more for the internal XAVC-I codec, not quite for ProRes RAW.

Sorry to say so, but from these lab results, I do not see the point of ProRes RAW HQ. It is not supported by DaVinci Resolve, you only have rudimentary controls (exposure & gamma curve, no white balance, no tint, etc….) in Premiere Pro and the files are noisier – to an extent, that even advanced noise reduction in DaVinci Resolve cannot save them. In addition, no highlight recovery options are available – one of the benefits that other RAW codecs (like BRAW for example) are offering."

I think I'll just stay using the internal XAVC approach and maybe jump between the RAW controls for the initial adjustments and then CSTs and nodes for the rest. But I will check out the BRAW from curiosity when it becomes available for the FX6 and 9.

Chris Young

 

[cyvideo]

The AXS Raw recorder for my Sony F55 snaps into a port on the back of the camera making it integral, no cables or wired external connections, and it unifies with the camera body. Because it's integral, I never thought of this an 'external' raw recorder, and apparently Red didn't either as the F55 was the first target of Red's lawsuit. Sony countersued Red for infinging on some of Sony's own patents and the parties settled. The terms of the settlement agreement are unknown but Sony's ongoing avoidance of such internal raw was voluntary by agreement, and as such Red joined the other members of the consortium informally known as 'the cartel,' Canon, Panasonic and finally Nikon among the camera makers.

So for Sony, after the F55 settlement there was never anything standing in the way of internal raw except their own voluntary compliance, which likely included not developing new external raw hardware. Enter BMD and others. But what constitutes internal raw has always been unclear at best.

To segue, Sony 16 bit linear raw is wavelet compression as was Redcode but along the line, Red switched to DCT and Sony released X-OCN.

In any event, it's been a while since I have worked with my F55 or 16b Linear Raw, but my recollection of it in Resolve was that it ran smoothly, with raw tab control on-par with BRAW. You could choose how to interpret the input as Slog2/Sgamut, Slog3/Sgamut3, Slog3/Sgamut3.cine, and possibly others. I don't remember color temperature controls working as well as BRAW (although it had them), but the other raw panel controls for black point, pivot, gamma, brightness and others were complete and fully functional in the panel. There were a few debayer options as well. Of course 4K 60 fps was the top level for the F55. BRAW 12K 72 fps on recent releases of Resolve with NVIDIA RTX4090 run smoothly.

Interesting observations TR, and noted. Good one.

Chris Young

 

[Tom Roper]

One other purely subjective and possibly biased impression for me comes from the comparison on the F55 of XAVC intra and 16 bit linear raw. It seemed that Alister Chapman (and now CineD) were saying XAVC was so good that there was little practical difference and so I didn't immediately move to get the Sony R5 raw recorder. When I finally did it was because the price fell on secondary markets and I was able to pick one up for very little. But my own impressions about 16b linear raw are a bigger improvement in IQ than I was expecting, so much so that I never used XAVC after that. The AXSM media was still expensive so I went on an obsession for snapping up whatever I could find when I found bargains. I also don't remember it giving the impression of being noisy, to the contrary it seemed clean. The most difficult to find component in the system was the AXSM card reader which I obtained from Adorama new. These current conversations have me inkling to bring out the F55 and shoot around a bit. The main reason I drifted away from it was the heft of having to shoot with PL glass from heavy duty supports was inconvenient. But I always agreed with Doug on this point, SLog2/Sgamut was visibly cleaner with color purity and DR better than SLog3/Sgamut3.cine, or gave me that impression if it wasn't.

 

[Tom Roper]

I was just reminded of one other comment I have that probably no one agrees with now, it's been so often repeated to shoot one stop over with Slog. All I can say, is not on my F55. Middle gray on SLog2 is 32% and SLog3 42% and that's what I used, always with good results. It made me wonder about something different in the follow-on cameras F7, FX9, FX6, dslrs et al, that is different, or changed, one obvious being that SLog2 went away.

 

[drboffa]

I believe the F55 had a different sensor from the F5, FS7, and FS5 (which all shared a sensor), which may explain the need to "overexpose" the latter but not the former.

That said, unless I'm shooting at High ISO and want the cleanest image possible I've found no reason to expose the FX6 one stop over. It's vastly cleaner than the previous generation of Sony cameras that I worked with (which admittedly did not include the F5 or F55). Sony's recommended levels work just fine at this point (again, at Low base ISO).

I did try ProRes RAW once or twice with the FX6 but it didn't seem worth the hassle.

 

[Tom Roper]

I think that's right. The F55 only had one base ISO setting, 1250. Moving the thumbwheel was called changing the EI, exposure index. You were not changing sensor gain with it. A change to EI in the camera was the same as a change in EI in post; the raw tab. With EI you were changing how you exposed. It worked like a viewfinder lut. If you lowered EI, the viewfinder was made darker, causing you to open the iris for more exposure. If you raised EI, the viewfinder went brighter causing you to stop down the iris for more highlight latitude. Gain was not affected. This is called ISO invariance, when a change in-camera has the same or approximate same change made in post.

So the notion that people preferred more or less exposure is one thing, but in some of the later Sony models a change to ISO was not a change in EI but was a change in actual gain, correct? (SLog3 Gain thus expressed by ISO numbers instead of decibels in other words.)

 

[drboffa]

I can't speak to the F5, but in the FS7 and FS5 you could raise/lower the actual gain. I'm pretty sure the FS7 also had a Cine EI exposure mode as well, although I only used that camera once years ago. The FS5 only operated on the premise of actually changing the gain (i.e., there was no EI mode). It was easy enough to "overexpose" by a stop or two using an external monitor and the appropriate viewing LUT (or a light meter or zebras etc) but in-camera ISO or gain changes were actual gain adjustments.

The FX6 currently allows you to shoot either way. Cine EI on the camera is the method you describe, although with two options for base ISO (Hi and Low). Flexible ISO is where you actually add/lower gain in camera.

I have yet to do testing to see what (if any) differences there are in practical terms on the FX6 between the two approaches, but one of these days maybe I'll get around to it.

 

[Doug Bee]

Wasn't the first version of Sony RAW, the one on the F55, always able to be processed in Resolve? I thought that was always the case. And it's fixed 3:1 DCT based 16 bit lin.

And THEN they later went to a wavelet style for X-OCN which is basically recode in a SONY package because they did an IP deal and agreed to not sue each other? This X-OCN came out much later. And isn't X-OCN and Sony RAW in fact different codecs? And then you now have BRAW and ProRes RAW external recording which are also different raw codecs?

 

[Tom Roper]

I can't speak to the F5, but in the FS7 and FS5 you could raise/lower the actual gain. I'm pretty sure the FS7 also had a Cine EI exposure mode as well, although I only used that camera once years ago. The FS5 only operated on the premise of actually changing the gain (i.e., there was no EI mode). It was easy enough to "overexpose" by a stop or two using an external monitor and the appropriate viewing LUT (or a light meter or zebras etc) but in-camera ISO or gain changes were actual gain adjustments.

The FX6 currently allows you to shoot either way. Cine EI on the camera is the method you describe, although with two options for base ISO (Hi and Low). Flexible ISO is where you actually add/lower gain in camera.

I have yet to do testing to see what (if any) differences there are in practical terms on the FX6 between the two approaches, but one of these days maybe I'll get around to it.

You can change gain in F55 Custom Mode, which is basically 709 mode or a subset of 709, i.e. Slog3/sgamut3.cine-like. You can also change WB temperature, and color matrix settings.

 

[cyvideo]

I think that's right. The F55 only had one base ISO setting, 1250. Moving the thumbwheel was called changing the EI, exposure index. You were not changing sensor gain with it. A change to EI in the camera was the same as a change in EI in post; the raw tab. With EI you were changing how you exposed. It worked like a viewfinder lut. If you lowered EI, the viewfinder was made darker, causing you to open the iris for more exposure. If you raised EI, the viewfinder went brighter causing you to stop down the iris for more highlight latitude. Gain was not affected. This is called ISO invariance, when a change in-camera has the same or approximate same change made in post.

So the notion that people preferred more or less exposure is one thing, but in some of the later Sony models a change to ISO was not a change in EI but was a change in actual gain, correct? (SLog3 Gain thus expressed by ISO numbers instead of decibels in other words.)

The other major differences are that the F55 runs the much wider DCI-P3 gamut than the F5's stock ITU Rec 709 gamut. The F55 has a P3 CFA, like the F65, and the F5 ran the stock 709 CFA. Plus of course the F55 runs a global shutter.

Chris Young

https://helpguide.sony.net/di/pp/v1/en/contents/TP1000756714.html

 

[cyvideo]

I've been tracking over on the Resolve forum since it first came out (as we thought it was a bug). I've attached a screen shot from one of my projects of the controls that get exposed. A couple of things I've found (at least for me),
- It only works with SLOG3/S-Gamut3.Cine (not the wider S-Gamut3) in an MXF container (from my FX6)
- you lose the ability to assign Colour Space / Gamut from the other menus for SLOG3/S-Gamut3.Cine NXF files (no big deal but why it was thought to be a bug)
- assigning other files to SLOG3/S-Gamut3.Cine does not bring up the RAW controls - eg you can't mark your A7Siii MP4 files to work with the raw tab.
- Colour Space and Gamma are always greyed out, and I can not change them (they are just for info which would makes sense).

These raw controls makes it is pretty easy to change the Colour Temp on a clip. You can select a preset, or dial it in using the colour picker on a white patch. Nice.

Correct on all counts.

It is only the FX6 and FX9 XAVC files that carry the correct 10-bit metadata information for RAW access. I believe more cameras will carry this info in the future.

I believe. I haven't tried it. It is that if you transcode your A7siii files to XAVC-I in Catalyst Browse, that Browse will add in the necessary metadata hooks to access whatever Sony's RAW needs to allow Resolve to access the files via the RAW tab.

Chris Young

 

[jmone]

I'm no expert on Catalyst Browse, but I just tried a A7Siii to XAVC-I converted clip in Resolve and it did not work bringing up the RAW tools. Not that I'd want to faff around with such a workflow anyway.

 

[cyvideo]

I'm no expert on Catalyst Browse, but I just tried a A7Siii to XAVC-I converted clip in Resolve and it did not work bringing up the RAW tools. Not that I'd want to faff around with such a workflow anyway.

Well now I believe it's not possible! Thanks for the heads-up. As you say, a bit of a convoluted workflow. And with no benefits to speak of.

Chris Young

 

[jmone]

I agree.