High Dynamic Range Concept?

N

notharrylume

Active member
I was thinking - on my morning walk -

Cameras now shoot at multiple ASA's, new ones as high as ASA 1600 or more
Cameras shoot at frame rates more than double 24 fps.

Why couldn't they combine the two characteristics to create video HDR? Shoot a 48 FPS frame at 100 ASA then shoot the very next one at ASA 1600, then back to 100, etc.

Use an intraframe codec. Then combine the ASA 100 frame with the ASA 1600 frame with an HDR algorithm into one HDR frame at 24 FPS with an added DR of 5 stops. That would give a camera with 8 stops latitude 13 stops, which is as good as claimed for color negative film.

It seems as if all the basic technology to do this exists today.

Why can't this be done?
 
Because we're talking about motion video here, so you'd end up with an interlace-style artifact spread across two or three frames.

I proposed this idea to some camera engineers a couple of years ago, and they said "yeah, we know -- but it doesn't work." And they explained why, and it makes sense...
 
Batutta said:
How about 3 cmos chips, each one recording 24fps at a different sensitivity?
Click to expand...

I did some experiments relating to this a while back. Did two identical static shots of a high contrast situation, one exposing to get the most shadow detail, the other exposing for highlights.
Then I layered them in FCP and used a luma key to take out any blown highlights from the shadow shot and replace them with the highlight one.
That was where the problem came in. I couldn't seem to get a smooth transition between the two exposures, although it looked really cool otherwise to have a 15 stop DR from my dvx. :)
And even if you figured that out, how do you process 24 fps like that and make it work for every situation?
 
We do HDR work a lot with photoshop at the design company I work for. It's nearly impossible for video, but not impossible for one shot. We tried this with a custom three camera rig last year. Kinda like a 3D rig. It worked o.k. The problem is that you have to adjust the exposure settings on all three cameras to the perfect balance. Totally improbable for any real type of shoot. We spent eight hours on one experimental studio lit shot. We captured the footage at several different exposure settings on all three camera's. Camera 1=Overexposed, Camera 2=Balanced, Camera 3 Underexposed. We then logged and layered the three shots on top of one another until the desired look was achieved. In my opinion as a graphic designer and cinematographer, it's a complete waste of time. But still a good idea.
 
In still cameras you can push and pull higher end pictures shot with a higher bit count, since a lot of data isn't seen but is still there. If a video sensor could record at those rates then the HDR can be faked without actual different exposures. It wouldn't work to have different ASA settings, even on different sensors, because the different noise would cause major artifacts. Obviously frame rate and aperture wouldn't work, either. The artifacts comment comes from my own not-so-extensive-but-still-above-average use of HDR mapping in Photoshop, which generally involves a lot of work by hand to make fix the many problems.

It's something I've thought about in the past, HDR video, but I think it's something that we'll have to wait quite a while for. Most of the market doesn't want it (or at least doesn't know they might want it) and the technology needs some work. I do expect it someday as the next frontier, after 35mm lenses are normal. Or the same concept could be used to create a non-rendered dynamic range, where exposure can be pushed and pulled a lot more than it is now, basically setting exposure in post.
 
Batutta said:
How about 3 cmos chips, each one recording 24fps at a different sensitivity?
Click to expand...

I hope you like your sensitivity through the roof.

There is only X amount of light reflecting of a subject and coming through a lens and onto a sensor. Three sensors would somehow need to use mirrors, prisms, or other optical devices to split the light among different sensors. The result is much lower sensitivity.
 
Ryan Patrick O'Hara said:
I hope you like your sensitivity through the roof.

There is only X amount of light reflecting of a subject and coming through a lens and onto a sensor. Three sensors would somehow need to use mirrors, prisms, or other optical devices to split the light among different sensors. The result is much lower sensitivity.
Click to expand...

True, but 3 ccd cameras have used prisms for many years. One CMOS sensor could be gained up, the other gained down and the noise averaged out amongst the three.
 
Batutta said:
True, but 3 ccd cameras have used prisms for many years. One CMOS sensor could be gained up, the other gained down and the noise averaged out amongst the three.
Click to expand...

But each ccd in the three doesn't have to do the same amount of work as a CMOS.

I imagine that although 3chip CCD cameras do utilize prisms, each one is responsible for acquiring it's own assigned RGB channel. This division of the 'work' I assume allows for greater sensitivity, as the photosites can be larger. 3 cmos sensors I'd imagine would have to pickup each RGB channel PER every chip... and thus would need RGB photosites on each chip and thus the photosites would have to share the sensor dimensions, and thus be smaller in size, aquiring less light over time which means less sensitive.

An image from a 1/3" 3CCD chip (generally) still has more sensitivity then if a prism split the light to three 1/3" CMOS sensors, since each CMOS has to do the job of a 3 CCD chip.

I'd imagine, if you had a 3 CMOS prism camera for HDR you would need a sensor that is either 3x as sensitive which means better technology or the sensor would have to have dimensions 3x that of it's previous size, as to get the photosites larger.
 
Barry_Green said:
This may be the single most brilliant thing I've ever heard proposed.

Batutta, you have earned your paycheck for the week. I am off to go lobbying now... :)
Click to expand...

Coming from Barry, that gives you some serious street cred.



And, it does sound like an idea with fantastic potential.
 
Batutta said:
True, but 3 ccd cameras have used prisms for many years. One CMOS sensor could be gained up, the other gained down and the noise averaged out amongst the three.
Click to expand...

If you are disregarding improved sensitivity and are just talking about artificial gain... what is the point then? If you are simply increasing the low sampling of photons hitting the photosites, you can do that in post by selecting highlight or shadow areas to increase or decrease gain. From my understanding, gain is just an amplification of a signal. You don't need three CMOS chips to do artificial gain.
 
I can't remember where I read this... but they are designing a single chip with three different layers, with three different programmable sensitivities. Kind of like Film or the Foveon chip, but every layer transmits light to the layer beneath and each has photosites for all three RGB colors, except the bottom layer. So there you have it... good engineers are very creative people. I just wish I could remember where I read it, sorry.
 
Batutta said:
How about 3 cmos chips, each one recording 24fps at a different sensitivity?
Click to expand...

I've read up on some papers that have achieved things somewhat similar to this, however they used 'Camera Arrays' instead of 3xCCD/CMOS.

I'll do a dig later tonight see if I can post something.

Anhar
 
You must log in or register to reply here.
Back
Top