Question about RED's HDRx: why don't people use it?

[Samuel H]

I see tests, but very few examples of HDRx used in real shoots. Why is this so?

The reason I'm interested in this is that I'm trying to develop a camera using the CMOSIS CMV12000 sensor, and it has less-than-spectacular DR. This issue can be solved by using one of the HDR modes that the sensor has. It creates the same image quality issues as HDRx (weird motion blur in the highlights), but none of the other problems (workflow, fps, bitrate, stay the same as in standard DR mode).

(This also explains why I didn't go over to reduser to make this question...)

 

[maarek]

Actually quite a lot of productions do use HDRx. I see it quite often in bigger budget films here in Finland, but don't really know how much is used in color grading. It's usually used only for select shots.

 

[S.Blanchardon]

Isn't there 3 different modes of "HDR" available for the CMV12000 ?

Taken from the Apertus AXIOM website :

"The image sensor supports three different High Dynamic Range (HDR) modes.

• One mode alters the sensors photosite response curve from a linear default behavior to a shape that is closer to a logarithmic response. This happens before the ADC so no digital data is sacrificed for the additional dynamic range.
• The second mode uses different exposure times for even and odd pixel rows. So two exposure brackets can be captured at the same time for sacrificing vertical resolution.
• The third mode is possible due to the high speed nature of the sensor allowing alternating exposure times with each recorded frame and doubling/tripling/etc. frame rate result in double/triple/etc. the amount of data to be captured.

Forget the second mode that reduce resolution, do you have motion blur in highlights when testing the first mode ?

Third mode looks like a "true" HDR mode where you have two different exposure from the same frame. But imagine it has the same defaults as first mode with really fast movement ?

Cheers.

 

[Samuel H]

Yes, there are 3 methods, but they all have the same problem: they create motion blur artifacts in the highlights, because they un-clip them by mixing the standard shot (where they are clipped) with one taken with a shorter exposure (where they are safe). The first method is the first one because everything happens directly at the pixel level (actually, even before taking the voltage to the ADC). It's not easy to calibrate, but it works great. Except for the motion artifacts, that's why I asked

 

[S.Blanchardon]

Could you share with us just one frame to illustrate this nasty effect please ?

Anyway, it's basically a show stopper that make the use of this sensor useless...or is it not ?

 

[Samuel H]

I'll ask cmosis to see how much I can share.

I don't think it's a show stopper, but it's a downer:
* It creates motion artifacts when there's movement; which sucks, given that the sensor has global shutter and would be an awesome option for shaky movement if it wasn't for this.
* It means that all that extra DR that you get has to be devoted to the very bright highlights. You can't expose most of your frame with this extra DR, because then you may get the fast shutter flickery look of Saving Private Ryan. So it's great for un-clipped sky and windows and such, but you can't put your skintones in that range. And there's not a lot of headroom in the shadows. Basically, it has to be lit like film: not great in low light, lots of DR on the highlights, not so much on the shadows, be ready to use fill light. Not a very bad thing, just not ideal, and different from what many people expect today.

 

[bgundu]

The truth of the matter is, many people do not know how to use HDR effectively in video. It's not as simple as providing a blend. The tools available in stills is not available in video. Some post facilities do it right by compositing properly. It's not a simple solution but as a post guy myself, I like having properly exposed footage in my back pocket when I need to plop it back in for some areas. Of course the two video streams have different shutters and effectively have different motion blur. You have to really plan for a HDRx shot. With the new Dragon, HDRx will not be an option.

 

[Samuel H]

The HDR footage can come blended out of the camera, so it falls down on the camera designer to make it look good. The user won't have to worry about workflow, just about exposure and motion blur artifacts.

As a post guy, would it bother you too much if the footage came with a baked-in knee in the highlights? Even the Alexa has a knee on the shadows, but they keep it log on the midtones and highlights. Given how DR is distributed on the CMV12000 with HDR mode on, I'd rather keep it log only for the midtones, and have a knee in the highlights as well as in the shadows. The codec would be 10-bit so you can still play with the highlights even if they're compressed with that knee.

 

[bruceallen]

Having linear with a knee is not much of an issue at all. We've been dealing with video cameras that have knees for ages.

As long as we can get to a linear image somehow by undoing the knee, we're ahead.

The issues with HDR mode on RED are that you have:
- 1/2 the max frame rate
- 1/2 the data rate per frame (and compression is an issue - it's not done in the most efficient way for HDR)
- The two exposures are taken at different times

The Cmosis sensor offers a few things that RED doesn't.

For example, I like the "alternating line" idea. I think it begins capturing both exposures at the same time?

Sure you will need to add a little motion blur to the underexposed layer to match the overexposed one... but that could be workable. Definitely easier than blending two exposures from completely different times.

The Cmosis sensor is a full 4x3 size sensor. Shoot with an anamorphic lens and you have two 4096x1536 images. That's enough resolution for me!

So... does anyone have any actual images from that sensor yet? I'm curious about the ISO performance and the color rendition.

Bruce Allen
www.boacinema.com

 

[Samuel H]

close

I have images, but I still have to hear back from cmosis about how much I can share

 

[Samuel H]

ok, so CMOSIS has cleared me to talk a bit more about this...

* ISO: not great, I've found base ISO to be around 200. Increasing analog gain only gets you to ISO 800, and it marginally reduces DR, it creates some noise, and it requires a completely new calibration of my favorite HDR mode (piecewise linear). Increasing digital gain is not an option given how narrow DR is to start with.

* Resolution: absolutely amazing, I'm seeing more detail with the CMV12000 than in the stills from my Sony NEX-5N (which shoots 4912 x 3264). This is probably because the evalsys had no OLPF, but then again, I saw very minimal aliasing/moire issues, even shooting charts. And the problems I've seen could be solved by using a softer debayering method. Since my camera (if I can ever make it work) will record RAW, I don't think a strong OLPF will be necessary, just change debayering method if you find moire.

* DR: definitely a downer. I measured a bit over 8 stops of usable DR in 12-bit ACD mode, and I'm an "optimistic DR measure" guy, Kholi would probably have said 6 or 7. Absolutely no NR applied in the shadows, but still, even if I can make those 8 stops work, it's very little.

* HDR:
I can distribute that DR as follows: 1 stop for the highlights, 4 stops for the midtones, 3 stops for the shadows
Then I can use PWL-HDR to expand that stop of highlights and, if I want to go overboard, I can make it something as silly as 8 stops. Yes, 8 stops, so 15 stops in total, but still only 4 in the midtones and 3 in the shadows (and Kholi might complain that they are 3 and 2). More reasonably, it can be 6, for a total of 13. This distribution of DR is the reason I said this sensor has to be lit like film: not great in low light, lots of DR on the highlights, not so much on the shadows, be ready to use fill light.
But the sky looks awesome. As long as you keep camera movement under control, that is: in those expanded highlights the exposure time is a lot shorter than for the rest of the frame, so you get motion blur artifacts. It's as if your subject was shot normally with 180º shutter, but you comp in a sky from a still and don't apply motion blur to it. Not a problem if you move the camera carefully, but given that the sensor has global shutter, I would have liked the camera to be free from motion artifacts.

Interleaved readout HDR mode gives you two 2048*3072 images (I don't know why the Apertus guys said it works in rows, I think it works in columns), and you can apply motion blur to one of them before you mix them. But still, it's a workflow-heavy option. PWL-HDR gives you one file that you can work with right away, I'd rather get that one working fine (not easy, I must say) and forget about all the others.

How does all that sound? Ready to shoot, would you pay $15K for a camera that records very sharp 4K RAW at 100fps with 13 stops of DR but has most of that DR in the highlights and with motion blur artifacts? And what if it only records ISO 200?
(That's my target price right now; but I'm still in the early stages of development -I have just tested the sensor and started working on the FPGA codes- so there can be great deviations from that)
(Also: don't get too excited, right now I rate my chance of success with this project at around 5%)

 

[S.Blanchardon]

Thanks for sharing your tests results with us !

It looks that except for resolution and global shutter, the other specs like base ISO and Dynamic Range are too poor for this sensor to be used on a real world camera, even with the HDR tricks that unfotunately introduce these motion highlights artifacts.

I'm sorry but I would never pay 15K$ for a camera with this sensor, better buy a Scarlet or an F5 for something around this price.

I know that Apertus is going to use this sensor for their Axiom camera... I was very excited at first but now I know that this project will never have an happy end, so to speak...

I wish you the best on this project Samuel, in the end it's going to be tough but fun to design a camera from scratch, but I sincerely doubt that you will manage to end up with a product that you could sell, sorry to be this blunt.

Looks that finally there is no miracle sensor out there, the CMV12000 looked like a good candidate but knowing it's true performance makes it a no-go IMHO.

I don't know of any other sensor manufacturers that could come with what we expect, unless it's custom built (I assume that's what RED is doing for their cameras and the new Dragon).

Does anyone know if for instance Sony is selling OEM versions of their sensors ? I know that the sensor on the Nikon D800 is made by them but was it built on order or was it an already existing product from an OEM range ? I'm dreaming out loud but it would be amazing to have an OEM version of the F55 sensor !

Anyway, it looks like it's going to take an other couple of years before we see the perfect and affordable 4K camera of our dreams. Maybe a Super35 4K Blackmagic camera will one day fulfill that wish ?

Cheers,
Stéphane.

 

[Samuel H]

Well, that was some honest feedback! Thanks a lot for that

First: I don't think the DR issue is a total dealbreaker. The artifacts will be there, but from HDRx tests I think an audience won't notice anything weird in any shot that wouldn't be ruined by jello on a Canon 1DC.
So far "image artifacts" is winning on the poll at the top of this thread, but it's just a couple of votes, and there are also some comments saying people happily use HDRx if they need the DR...

And then: Sony has a very limited range of readily-available sensors, nothing bigger than 2/3" IIRC. They sell bigger sensors to Nikon and Pentax, but won't talk to you unless you can put a few million dollars in the table and sign a contract that assures them you'll buy A LOT of sensors. And even then, I doubt they'll give you the F55 sensor. Maybe the one on the NEX-5N, but that one has to do line-skipping to record video.

 

[TrueIndigo]

"I rate my chance of success with this project at around 5%" - you are controversially sensible, Samuel!

Without naming any names, have you seen any other sensors which would be more suitable, or is this Cmosis unit the current best (relatively affordable) sensor available for independent camera makers? Must say, the low light performance and HDR-augmented DR sound worrying, but if anyone can fine tune the output, it's you! All the best with your project.

 

[Samuel H]

Given that it has to be at least 22mm wide and it has to work at >24fps...
The only other alternative I know about that meets these criteria and is available for third parties is the Panavision/Dynamax DYN0174. It's just sampling, it's probably much more expensive, and it probably has the same issues (I can't find it right now, but IIRC they rated it at 10 stops of DR in standard mode, just like the CMV12000) (edit: it's in the comments here: http://image-sensors-world.blogspot.com.es/2012/11/panavision-announces-174mp90fps-global.html and there's some optimism too: maybe those 10 stops are in global shutter mode, and it's a bit higher in rolling shutter mode; or maybe not: very little info around so far).

Apart from that, there are a couple of 1" or smaller sensors that would be fast enough and have better DR, but it's too late for that, BMC owns that market.

 

[Samuel H]

And just to restore some optimism...

These are two 100% crops, first from a still image shot with a Sony NEX-5N (which shoots 4912x3264), and then from the CMV12000 (without OLPF, I think).

No sharpening or anything in the CMV12000 image: just debayer (using "smooth hue transition" method), quick color balance, adjust contrast.