The limits of raw…. or...

legrevedotcom

legrevedotcom

Veteran
…maybe the limits of my computer really ;)

Anyways, have been excited about the MLV thing they've got going and decided I wanted to try an interview situation today to help a kid making a school report.

So I set it all up and ended up having to answer questions for around 7 minutes.

That gave me around 10.000 frames to handle.

I was using my MBP with ssd drive and 16gb ram. My workflow with raw is that I go through Adobe raw because I have some simple settings that I like…. this time, I didn't even want to wait for CC to load up the dng files haha.
I started chunking 500 frames together and do the basic grade and export to tif….. still took ages. So in the end I just quit that damn thing and we redid the interview in Canon normal :D


My conclusion…. raw is good for small takes only and even then you still have to save up time and space for handling the files. Raw is not for everyone, but it is most definitely not for long interview sessions… EVEN with MLV module :)
 
...Or the limits of your workflow. :)
Interviews are hardly prime candidates for raw shooting anyway. I am not sure why you bothered with the tif step? Why not output in a compressed video format?

Logistics with HD raw are largely overstated though (not by you; rather by the internet as an abstract).
I recently dp'd a short, all done with ML raw (non-mlv). The thing was shot in 2:1 or 1920x960. Got 710gb dng files for 163 minutes of material. Passed this through DNG converter and got 399gb of losslessly compressed dng files. That's a reduction to 56% of the original datasize. Not bad? But wait, there is more. 399gb for 163 minutes roughly equals a bitrate of 330mbps. At full resolution (1080p) this would be around 370mbps. For comparison, 10-bit DNxHD 4:4:4 1080/24p is 350mbps. So I am getting the full possible quality (14-bit raw) with no loss whatsoever at a bitrate comparable to a popular 10-bit intermediate compressed codec.

The bonus is that you don't need fancy raid storage setups or SSDs. The footage will play like a charm (i.e. realtime) from any decent defragmented 5400rpm hard drive (tested that with 3 different drives).

And I am still perplexed why would people inflict ACR upon themselves for raw footage (well, the camera specific noise profiles probably). I did some tests and my eyes see zero difference in quality between ACR and Resolve 10. I am not saying there isn't any, but I am not seeing it. And Resolve is infinitely faster and free.
 
cpc said:
And I am still perplexed why would people inflict ACR upon themselves for raw footage (well, the camera specific noise profiles probably). I did some tests and my eyes see zero difference in quality between ACR and Resolve 10. I am not saying there isn't any, but I am not seeing it. And Resolve is infinitely faster and free.
Click to expand...

You can push a file around FAR more in ACR. Quite incredible really. Resolve doesn't even come close. However, because ACR is always adjusting exposure, you will always get in flicker too. Thus it is basically unusuable anyway. What people have asked for is that it ingest all the frames at a single set of adjustments, rather than compensating for each frame. This hasn't happened. Real life workflow = Resolve. You don't even have to transcode beyond the cinema DNG's if you don't want.
 
* AE/ACR will transcode DNG footage at around 2fps; if you can't do it overnight, it's not an option, even for small-ish projects
* The flicker thing can be avoided (you can either leave untouched some specific sliders that create this problem, or use an older debayering process)
* Resolve doesn't have any of these issues, it should work in realtime if you have a good GPU
* Resolve is as sharp as ACR, but I've read comments that ACR gives you better DR because it can dig deeper into the highlights; I'm not a Resolve user so I can't comment much on this
 
I use Lightroom a lot, all my stills (including my scanned film stills) go through LR. It can certainly do some amazing things. Highlight extrapolation is pretty nice. So are chromatic aberration removal, profile based noise reduction, local contrast control, etc.

But for quick debayering + primary corrections of video [and, frankly, that's probably all one is going to do in ACR/LR with raw video; the HSL sliders are the only secondary adjustments you can do in it], I didn't see any benefits, and I did push files around in both ACR and Resolve, within sensible limits. The difference in highlight recovery with my test files was negligible; in fact, I often wondered if I am imagining it. Also, didn't notice anything worth mentioning in the blacks. Now maybe there is a difference if you push files really, really hard, but how often do you need this in primaries? Maybe when applying some specific look, and you'll probably be doing this in the grade (after ACR) anyway. NR in ACR/LR is really nice as it is profile based, but it is spatial NR only. And I find with video you should crank it significantly higher than with stills. I didn't have the patience to test if thoroughly with video though. NV is doing a really nice job with this anyway.

One cool thing with Resolve is that presumably decompression is happening on the CPU while debayering is on the GPU, so there is practically no difference in processing speed between uncompressed and losslessly compressed dng files. I get 50-70fps on export, and I am not sure this is GPU/CPU limited (haven't bothered checking), could be file system/drive speed limited. No need for fancy GPUs either, something cheap like a used gtx570 is certainly more than enough for realtime playback.
 
ACR does "better highlights" because it processes it quite smartly. But the same data is there in the Resolve output too, it just doesn't combine the channels intelligently when bringing highlights down. But acr does introduce flicker with highlight and shadow controls anyway. For example, you can use the same highlight recovery for 8-bit jpegs with ACR and it works wonders for those too, quite funny.
 
You can simulate ACR-like highlight extrapolation (rebuilding highlights from at least one unclipped channel) in Resolve with the RGB mixer: increasing the healthy channel for a clipped channel will "magically" unclip highlights. I rarely see the need to do this though, even though I do expose hot for cleaner signal.
 
yoclay said:
You can push a file around FAR more in ACR.
Click to expand...

This is interesting. Does the image break down sooner in Resolve? I've never played with both programs in this way on identical DNGs to know this.

I wonder why BM can't duplicate what Adobe does in ACR
 
Steve Kahn said:
This is interesting. Does the image break down sooner in Resolve? I've never played with both programs in this way on identical DNGs to know this.

I wonder why BM can't duplicate what Adobe does in ACR
Click to expand...

There is just simply not as much highlight recovery, but also I can pull selective contrast in ACR that would require me to practically go into the secondaries of Resolve in order to get them. The information may be there, but the tools are not. The algorithms in ACR are very powerful. Playing with a few sliders for several seconds one can achieve what might take minutes and sometimes hours in Resolve.

However, it remains a moot point, you can't really use most of these sliders because of the flickering they engender in the output.
 
yoclay said:
You can push a file around FAR more in ACR. Quite incredible really. Resolve doesn't even come close. However, because ACR is always adjusting exposure, you will always get in flicker too. Thus it is basically unusuable anyway. What people have asked for is that it ingest all the frames at a single set of adjustments, rather than compensating for each frame. This hasn't happened. Real life workflow = Resolve. You don't even have to transcode beyond the cinema DNG's if you don't want.
Click to expand...

You can grade one key frame for a scene or clip in ACR and copy the profile to all the frames for all shots for the scene to get consistent results rendering to tiffs or jpegs or video encoding through AE. It works just fine.

But if you are interested in real time speed, ACR via AE or Speedgrade, Pomfort Cliphouse or Resolve are the only reasonably high quality options.
I think it is a mistake to treat cinema DNG as an editable video codec. Hollywood standard practice for all raw formats is to transcode to proxies and edit those, relink to raw for final grades, or transcode a corrected one light to a high quality intermediate codec like Prores, then edit and finish grade those.

I prefer the debayer results from either Raw Therapee or Capture 1 over the less sophisticated algorithms in Resolve and ACR that can run in near real time. Both of these run in a 96 bit workspace instead of 16 or 32 bit with advanced state of the art debayer algorithms and processing tools. But you are talking processing in seconds per frame rather than frames per second. I think of it as "lab" work that you have to plan time for in your workflow. Best reserved for problem shots or fine art quality work where time constraints are not a problem. For the HDR transforms I like to us, average time to process a 3.5mb 2K 2048x1152 frame in either of these on my 5 year old Core 2 duo laptop is between 10 and 20 seconds. Got components for a new Haswell i7-4770K workstation with a GTX760 coming next week. That should speed things up quite a bit, but it still won't be anywhere near real time.

The Pomfort software is Mac only for now, but they are supposed to port it to PC sometime in the future. For those of you working on Macbooks, it is a great DIT solution and you can get it from Pomfort, or the slightly different Lightpost version from the Digital Bolex store. Both work with any current cinema DNG standard camera footage. They give a choice of debayer algorithms from quick and dirty to sub real time high quality. Much simpler and quicker to learn than Resolve.

http://pomfort.com/cliphouse/

http://www.digitalbolex.com/product/lightpost-software/
 
Last edited:
You don't even have to copy the settings from one key frame to all the others in After Effects. The settings from the first DNG frame in a sequence will automatically be applied to the entire set of DNG frames. I hate how slow ACR is rendering in After Effects, but damn I can't even come close to the same quality with anything else.

I have used Cineform, Resolve 10 Lite, and ACR via After Effects to process ML Raw sequences. Cineform is the fastest by far but the quality is not good. Resolve is much faster than ACR but still the price is less detail, demozaic artifacts, less dynamic range, incorrect colors, and no proper white balance adjustment.

For me there is no easy way around. Everything that is fast, pales in comparison to the quality of using ACR with After Effects.
 
Last edited:
Celluloid Dream said:
You don't even have to copy the settings from one key frame to all the others in After Effects. The settings from the first DNG frame in a sequence will automatically be applied to the entire set of DNG frames. I hate how slow ACR is rendering in After Effects, but damn I can't even come close to the same quality with anything else.

I have used Cineform, Resolve 10 Lite, and ACR via After Effects to process ML Raw sequences. Cineform is the fastest by far but the quality is not good. Resolve is much faster than ACR but still the price is less detail, demozaic artifacts, less dynamic range, incorrect colors, and no proper white balance adjustment.

For me there is no easy way around. Everything that is fast, pales in comparison to the quality of using ACR with After Effects.
Click to expand...

This is my viewpoint too. Fast debayer = mediocre quality. I've seen some really nice results from Resolve but it is a completely different paradigm with a steep learning curve. Resolve 10 did take a big jump in debayer quality, but still not up to ACR or the more sophisticated stills processors like Capture 1 or Raw Therapee.
 
Resolve is fast, but like others have said, the results from ACR are just stunning.

I make sure "small hacks" are disabled in ML (whether that helps or not, I dont know, it seemed too) and I use a Deflicker pluggin which excellent for ridding footage of any flicker.

ACR noise reduction and sharpness together with the highlight recovery makes it worth the extra effort.

I have to say... working with ML_Raw is flippin awesome....I love the extra dynamic range, and the latitude when grading... mixed with the beauty of Full Frame Goodness, its made all of my lust for next gen camera's disappear..I'll be using this for some time!! :D
 

Attachments

  • bmpcc400%2.jpg
    bmpcc400%2.jpg
    94.8 KB · Views: 0
  • BMPCC400%debayer.JPG
    BMPCC400%debayer.JPG
    56.1 KB · Views: 0
Last edited:
@cpc
How did you manage to get losslessly compressed DNG files reduction to 56% of the original datasize ?
I have got only about 30% reduction using this DNG converter.
 
gideon said:
@cpc
How did you manage to get losslessly compressed DNG files reduction to 56% of the original datasize ?
I have got only about 30% reduction using this DNG converter.
Click to expand...

This is probably down to subject matter. DNG lossless compression is a zip variant, iirc. A lot of varying detail in the frame should compress less.
 
You must log in or register to reply here.
Back
Top