Magi Lantern and RAW workflow doubts

The problem is that you're more prone to losing everything in the event of a failure rather that just a portion. I much prefer spanning to a single file.

Gary,
What about Ninja2? Doesn't Ninja2 save it in one big file? Or does it do file spanning?
 
While the RAW news is certainly an awesome achievement, I think most users are getting WAY ahead of themselves by getting excited about RAW. Even in best the cases, you'd probably just want to save this for extremely demanding, high-detail b-roll. It's just so big and so processor-intensive. Even the BM Pocket Cam has compressed RAW, which should be easier on your machine than the RAW DNG's this ML hack will create.

Raw is heavy on storage, but actually easier for processor to deal with.
 
Gary, What about Ninja2? Doesn't Ninja2 save it in one big file? Or does it do file spanning?

The Ninja-2 saves one file per take. And, yes, I've lost entire clips because of this. I had a faulty dock when I first bought it, but Atomos worked with me and we got it replaced and it's been rock solid since then.
 
RAW is just different. In terms of processing, it takes away the decoding step, and in turn adds a lot of new steps: debayering, denoise, sharpening, etc. If CS7 has a GPU-accelerated way of dealing with DNG files, this may be less of an issue soon. But I wouldn't count on that, and right now, RAW is awesome, but painful.
 
While the RAW news is certainly an awesome achievement, I think most users are getting WAY ahead of themselves by getting excited about RAW. Even in best the cases, you'd probably just want to save this for extremely demanding, high-detail b-roll. It's just so big and so processor-intensive. Even the BM Pocket Cam has compressed RAW, which should be easier on your machine than the RAW DNG's this ML hack will create.

I'm excited about the possibility because I shoot low-budget commercials on occasion and I was thinking about adding either a BMCC or the Pocket Cam. I'm keeping my Pocket Cam pre-order (if I'm lucky enough for an early delivery I can likely flip it with no loss) but this development means I'm going to hold off on searching for a used BMCC ... for now. I keep flip-flopping on the Pocket Cam and 5d3 is creating more doubt. Everything I shoot is very documentary style and I think this will be a great camera for my workflow. I find the C100/300/FS100 too big to walk around with on a monopod all the time.

I'm still amazed at what the ML crew is doing with all the features they're adding, I've donated to the effort a number of times. I hope Canon doesn't make them an offer they can't refuse -- just to make Magic Lantern go away...
 
RAW is awesome, but painful.

And also, for a lot of situations, completely unnecessary. The H264 codec, while not ideal, is perfectly well-suited for most productions (especially for the web). Far more important, as always, are proper lighting and proper exposure. RAW is a nice luxury to have, but it certainly isn't going to make a better film.
 
DNG raw is exactly what the name says; a digital negative. It is not meant to be edited directly in an NLE, but corrected in first pass debayer for color balance, gamma, and exposure then transcoded to video prior to edit. Even at the high end this is how it is done. You can apply log gamma curves to give grading room in the transcoded video file, or you can link back to the original DNG clips via XML for final grading from the originals after editing is done.
The biggest advantage of raw is the ability to recover highlight and shadow values that would be lost with a video original. That is done in the first pass debayer along with accurately correcting exposure, white balance, and grayscale. Depending on your computer and how sophisticated the debayer processor is it can take significant time for transcoding "dailies".

Working with Ikonoskop samples in Raw Therapee or ACR at highest quality debayer settings can take from 1/4 second to 8 seconds per frame on my laptop depending on how much processing is applied to get a look.

With RAW you have to do your "lab" work first just like film.
 
Based on what? Your gut? Do you understand the bandwidth requirements necessary in order to accomplish spanning?

File spanning works on h.264 so why not on raw? They are individual frames anyway so basically they are already spanning.

And those who are getting 1fps only, have you tried checking multiprocessing on in AE?
 
File spanning works on h.264 so why not on raw? They are individual frames anyway so basically they are already spanning.

Two misconceptions here...first, for you first question to be valid, the Raw file would need to be the same bitrate as H.264, which it is not. I guess you don't understand what "bandwidth" means? H.264 on the 5D (IPB) is about 50mbps of bandwidth needed to keep the recording going. For spanning, you need twice that, or 100mbps. Still not even close to the speed needed to write Raw data at a minimum of 24fps.

And they are not individual frames. It's a single file that has to use a utility to split it up into CinemaDNG.
 
...And they are not individual frames. It's a single file that has to use a utility to split it up into CinemaDNG.
Where is this "splitting up" taking place? All the samples I've seen have been zip folders of *.dng files. I just assumed they were going to cf card as individual *.dng files and never understood the issue with spanning.

Curious as to what the actual output is on the card?

Cheers,
Pete

EDIT : I just found it here - http://www.magiclantern.fm/forum/index.php?topic=5240.375 a1ex discusses the difference between silent DNG pics and continuous recording of RAW in a single file in post #387. It sends you here for that discussion - http://www.magiclantern.fm/forum/index.php?topic=5247
 
Last edited:
Two misconceptions here...first, for you first question to be valid, the Raw file would need to be the same bitrate as H.264, which it is not. I guess you don't understand what "bandwidth" means? H.264 on the 5D (IPB) is about 50mbps of bandwidth needed to keep the recording going. For spanning, you need twice that, or 100mbps. Still not even close to the speed needed to write Raw data at a minimum of 24fps.

And they are not individual frames. It's a single file that has to use a utility to split it up into CinemaDNG.

I don't understand why you think you would need twice the bandwidth.

All that would happen is when you set the mode, a DNG file header would be created for the first file, when you hit record the data would be written to that file, an EOF flag would be written, then it would repeat, new file header, data, EOF, and so on. In fact, since the number of bytes per frame is already known, it would be relatively easy to do.

Hell, you wouldn't even need to write a DNG header, you could do that afterwards, sort of like what they are doing now, except instead of 1 large file, you have multiple smaller files.

It isn't like you would be recording 2 files simultaneously, there is no need for twice the bandwidth.
 
All that would happen is when you set the mode, a DNG file header would be created for the first file, when you hit record the data would be written to that file, an EOF flag would be written, then it would repeat, new file header, data, EOF, and so on. In fact, since the number of bytes per frame is already known, it would be relatively easy to do.

What part of this:

And they are not individual frames. It's a single file that has to use a utility to split it up into CinemaDNG.

Are you unable to comprehend? The ML hack currently does not write out CinemaDNG to the CF card. It's a raw dump to a single file per 4GB limit, and you copy that file to your computer and use a provided utility that takes that raw dump and converts it to individual CinemaDNG frames. Were you not aware of this?

Did you not see this either?

I'm not saying extra bandwidth is not needed for that brief moment when you (may) have two files open at the same time. All I'm saying is that if you have those extra requirements for spanning, you have them too for single-frame-file writing.

I don't know this for certain, but I don't think it does because the process should be fast enough that all those files are written sequentially. No need for starting the write of a second frame while finishing up the first.
 
From what I understand it doesnt get converted to cinemaDNG just DNG sequence. IF they could put it in cinema DNG I think it would be game over. There are plenty of easy workflows for cinema DNG. I am currently editing/grading raw CDNG in resolve on my Mac pro (extra GPU and raid) in real time (BMCC and A-CAM DII footage). With resolve 10 there will be no need for anything else really if you dont want to mess with it. Im so tired of people complaining about raw and how hard it is. If you dont want to deal with it then just let those of us willing to deal with it to ourselves. I have been in a uncompressed all raw workflow for a year and I have not lost my mind or gone crazy. You know there are painters that still use actual paint not a cintiq w/ photoshop even though its way more time consuming and expensive.
 
From what I understand it doesnt get converted to cinemaDNG just DNG sequence. IF they could put it in cinema DNG I think it would be game over. There are plenty of easy workflows for cinema DNG. I am currently editing/grading raw CDNG in resolve on my Mac pro (extra GPU and raid) in real time (BMCC and A-CAM DII footage). With resolve 10 there will be no need for anything else really if you dont want to mess with it. Im so tired of people complaining about raw and how hard it is. If you dont want to deal with it then just let those of us willing to deal with it to ourselves. I have been in a uncompressed all raw workflow for a year and I have not lost my mind or gone crazy. You know there are painters that still use actual paint not a cintiq w/ photoshop even though its way more time consuming and expensive.

My friend's home PC gets 24 frames of smooth playback in resolve, totally makes me jealous. I get 14 on my retina.
 
My friend's home PC gets 24 frames of smooth playback in resolve, totally makes me jealous. I get 14 on my retina.

Yeah I notice most of the people complaining about frame rates are on a retina MBP or something with similar specs.

For raw, Hackintosh is king (unless Apple starts caring about pros again).
 
Yeah I notice most of the people complaining about frame rates are on a retina MBP or something with similar specs.

For raw, Hackintosh is king (unless Apple starts caring about pros again).

Im not complaining at all. It's fine, I had no intent to cut raw and not sure what the benefit of it is when you're in an environment that requires you compress versions of an edit for clients to watch.

I would like it just for my personal use, when shooting random tests or scrubbing footage but it hasn't been an issue at all. I go right out to ProRes 720 or 1080 to cut, and only return to CinemaDNG for color.

Stopped using Hacktinmacs six years ago or so. Just need something that works as is, personally.
 
Back
Top