How can a single-chip camera claim 4:4:4 color?

[Mark Sullivan]

How can a single-chip camera claim 4:4:4 color?

With a Bayer-pattern filter, it takes four pixels to get complete color information, right? So why is that any different from the chroma sub-sampling in normal Mini-DV compression? Why isn't 4:1:1 the absolute best that any single-chip camera can do, at least at full resolution?

I'm willing to belive there's something I don't understand about this, or there would be no point in using 3-chip sensor blocks for Mini-DV cameras since the output is going to be 4:1:1 anyway.

Can anyone help me with this apparent contradiction?

[krestofre]

I'm not too up on the technical, but I believe it's the difference between a CCD sensor and a CMOS sensor.

[Mark Sullivan]

That doesn't make any sense. No mattter how the underlying photodectors are made, a single-chip camera does not get full color information from a single pixel. It either gets R, G, or B and an interpolation algorithm combines these with the signal from adjacent pixels to interpolate the color.

My point (of confusion) is that the output of this interpolation comes up with a color value for each pixel, but wouldn't the overall bandwidth of the color information still be 1/4 of the total image bandwidth?

[Graeme_Nattress]

I hope we're not claiming 4:4:4 at full sensor resolution. That would not be the case for us or any other single chip bayer pattern sensor camera.

In some modes we deliver full 4:4:4 RGB though - "2x over-sampled HD in 4:4:4" because we're 2x oversampling, using 4 pixels to make up either 2k or HD. We can do this because we have many more pixels than we need for most resolutions.

Now lets look at full resolution or cropped modes, say 4k. At that resolution, it needs to be bayer de-mosaiced to be visible as RGB. There are many ways to de-mosaic, but good ones tend to give > 70% of the linear resolution out as real resolution, so 4k 4096x2304 gives out > than 2867x1612 real full resolution, although it will appear in an image of the full resolution of 4096x2304. It's the difference between measured resolution and pixel resolution. You can dub DV to HD and you will get a measured pixel resolution of 1920x1080, but it still has DV resolution, if you get my analogy. Now this is the same for all of use single sensor cameras, and Arri do a very good job of explaining it on their website. A bayer pattern image of 1920x1080 leads to about full 720p resolution, etc.

Also, the optical filtering in camera does some degree of spreading the colours over the pixels of the CFA, which doesn't happen in the decimation of 4:4:4 to 4:2:0 in a DV camera's processing.

Now, on a large chip, at high resolutions, bayer pattern will look better than you'd think from the above explanation, as lenses and reality limit the resolution coming into the sensor. You just have to look at the images from a top end Canon or Nikon DSLR to see that although bayer sounds bad, in reality, with good algorithms, it isn't.

Now onto DV....

NTSC DV is 4:1:1 but that's not a good analogue to bayer pattern. http://www.lafcpug.org/Tutorials/basic_chroma_sample.html

However, PAL DV 4:2:0 is. So for NTSC yes, you'd need 3 chips because of the pattern of the chroma sampling. But 3 chips at these low resolutions give advantages - some oversampling and they tend to be bigger chips. 1 chips cameras tend to have very small chips. As the resolution gets higher, any issues with bayer pattern rapidly diminish, and having one, nice, big chip becomes tremendously advantageous. Or another way to look at it, is the little Sony 1 chip HDV camera has the same resolution as it's big 3 chip brother, and some think it looks better too.

I hope that helps you,

Graeme

[Haakon]

I hope we're not claiming 4:4:4 at full sensor resolution.
I think the original "teaser" website certainly implied that ("2540p. 4:4:4." - though they never explicitly said they'd do them together...), but the wording in the new one does serve to clarify that notion. I think even 2K at 4:4:4 is amazing, and I doubt many of the "independent" crowd is going to be shooting 4K for awhile anyway, so it's not that big of a deal. The format options page (http://www.red.com/product/format-options/) still doesn't list 4:4:4 as an option for 1080, however, and I think there is just a little general confusion around because of some of these inconsistencies. That being said, I don't believe for a moment that the people behind this project are going to allow the camera to do anything short of amazing things, and I'm confident that the priority of image quality is (and has always been!) #1. Keep on truckin!

[Graeme_Nattress]

At 4k it's not 4:4:4 or any x:x:x. It's raw bayer pattern and it is what it is. However, yes, at 2k scaled it's 4:4:4 in every meaning of the notation and oversampled too.

Remember, we're in development and things can and will change. I hope this explains the issue though.

Graeme

[stokestack]

Thanks for the 70% figure, Graeme. I've been wondering what the approximate "real" (de-Bayered) resolution is for a while.

[Graeme_Nattress]

I think I got that 70% from a BBC R&D paper, but the nice thing in reality is that it actually looks better than that! And remember, > 70%, so that number can be higher although it can never reach 100%, and it all depends upon the algorithm used. Some produce higher resolutions, but have higher noise or deal with colours worse, and some produce softer images with less artifacts or none at all.

Graeme

[Mark Sullivan]

This discussion clears things up tremendously.

I remember when Nikon added the optical low pass filter in front of the 2MP sensor in the D1. It made an incredible difference. I did lots of testing on that camera. I'm excited to learn the Red has such a filter.

[Graeme_Nattress]

Every camera needs an OLPF. It's vital. Nikon usually make their's too weak, and get some extra resolution at the expense of aliassing. Canon usually get their's just right. I've never seen any nasty aliassing from them. Some high end medium format Digital cameras have no OLPF at all, and they alias a lot, especially on some architectural or clothing / fine weave shots.

[Jarred Land]

I think im going to make this a sticky... thanks again Graeme for shedding some light, honestly and openly. You should be commended.

It must be great to be in a situation where you have more pixels then you need rather than not enough pixels. Having not enough pixels to hit a resolution is where you start to get in trouble, as you not only need to interpolate actual pixel resolution, but the color values within those artificial pixels.

[Stephen W]

The question of how much "actual" resolution you get from a Bayer mask came up in a thread before. At the time I posted that it was 70% (ish), but remember that someone who was obviously more involved with the technology than I am said in real terms they were seeing nearer 90%. Not sure who it was though.

[Greg Lowry]

Coincidentally, yesterday I asked Dalsa about maximum RBG output resolution from the Origin. I received this response: "We do our highest quality bayer reconstruction to 4096 x 2048 (4046 x 2048 if you cut out the 50 black pixels) at 16bits per RGB pixel."

So Dalsa can achieve 100% of sensor resolution from it's reconstruction algorithms. They use a single CCD with Bayer striping.

I was surprised. So much for generalizations.

[Mark Sullivan]

I think everyone obtains 100% of sensor resolution when they demosaic the output of a Bayer-pattern sensor. This just means they get back the same number of pixels that they put in. The point of the number: 70%, 90%, etcetera is how good the reconstruction is compared to a real 3-color per pixel sensor (like a 3-chip sensor block).

I'm sure the success of the demosaic has a lot to do with the image. This may be one reason the figure of merit varies so much depending on whom you ask. There's probably not one perfect algorithm for all images. But there are also not a lot of images where the color bandwidth in the image truly needs to be as high as the luminance bandwidth.

The one application where you need all the color bandwidth you can get is chroma-key. I could see keying and compositing using the 4K raw output of the Red One, even though I would eventually scale down to 2K for the rest of the workflow.

[Graeme_Nattress]

I don't think chroma key is a big issue, as with the right software you can re-create 4:4:4 chroma from 4:1:1 see http://www.nattress.com/Products/FinalTouch/FTGChromaSharpen.htm for one of my plugins that does this.

Biggest thing that kills chroma key is not chroma sampling, but noise, poor compression, and above all, poor lighting.

Graeme

[Greg Lowry]

Mark, that's a rather off-hand dismissal, because DALSA is the only company I know of that claims to be capable of projecting true 4K RGB images from a 4K sensor (apparently it takes 4 x 4:4:4 HDSDI connections for data transfer). As Bayer striped sensors appear to be the way of the immediate future, we all need to understand their capabilities better.

[Graeme_Nattress]

16bit at 100% resolution - I'll believe that when I see it. That would imply a >96db SNR (given their chips aroun in the >70db SNR stats according to the Dalsa web site), which I don't quite believe, and 100% RGB reconstruction seems to imply magic.

[Barry_Green]

Graeme, thanks for the detailed explanation and taking the time to do so. We're glad you're here!

[Greg Lowry]

Graeme, rather than dismissing it as impossible, perhaps it would be useful for you to contact DALSA directly to get the facts yourself because it's ultimately an important issue and has relevance to RED. (If DALSA can do it, perhaps, RED can too?) I wanted to understand more about the various implementations of Bayer filtering and reconstruction, and especially about the claims of "real 4K" resolution after reconstruction because I too believed it wasn't possible, but that was DALSA's reply. I'm always eager to be enlightened by brighter minds, including yours, but facts are the best way to do that. If the info I received isn't convincing, then by all means make your own enquiries of DALSA. I suspect the board is interested in what you find out. I know I am.

[Graeme_Nattress]

Well, I've read just about every scholarly paper on de-mosaicing there is, and nobody can get 100% RGB equivalent resolution out. Given that Arri correctly, IMHO, compare RGB to bayer resolution, I'd have to say that perhaps Dalsa didn't understand your question to mean quite what we think it means. And if Dalsa are doing something not covered in the vast knowledge of what bayer can do, they're hardly likely to tell me what it is they're doing...

Or you can read Dalsa's literature that says : "Compared to a monochrome
sensor with the same pixel count and dimensions, the mosaic
filter approach lowers the spatial resolution available by roughly
30%, and it requires interpolation calculations to reconstruct the
color values for each pixel."

http://www.dalsa.com/dc/documents/Image_Sensor_Architecture_Whitepaper_Digital_Cinem a_00218-00_03-70.pdf

[Greg Lowry]

If you care to phrase a question on the subject that is free of ambiguity, I'll present to DALSA and post the response. If not, I'll drop he matter on this board, but it seems a subject worthy of further exploration with the companies making the cameras.

Today, I also heard from Radu Corlan at Vision Research regarding the Phantom 65 (which uses a 4K bayer-striped CMOS sensor). He wrote: "The Phantom 65 will output full-resolution images (up to 4096x2440)". When I ask for clarification regarding whether he meant "viewable" reconstructed RGB 4:4:4 images, he replied: "yes, the kind you would show on an IBM monitor or the 4k sony projector."

[Graeme_Nattress]

Yes, you do get an image out with 4k pixel resolution, but if you were to measure the resolution in terms of what is the finest detail that image can reproduce, the number is lower.

This Arri article explains it rather well:

http://www.arri.com/news/newsletter/articles/09211103/d20.htm

Graeme

[Greg Lowry]

I've very familiar with that article and the method ARRI uses. Some very experienced cinematagraphers who have seen the DALSA 4K images projected have said they're spectacular and certainly the best of what's available at present. For me, what's on the screen is what ultimately matters, not a white paper or theory.

I still don't understand why you're so dismissive, but I'll drop the matter.

[Graeme_Nattress]

Yes, the Dalsa images are lovely. I don't see what the issue is when even the Dalsa paper I pointed out confirms precisely what I'm saying. I just thing there's a difference in language between pixel resolution, and the measured amount of detail in the image, for which all bayer sensors have the same limitation, and by which all reconstruction algorithms are so limited.

Graeme

[Greg Lowry]

Sure, I accept the technical difference that you describe. In the end the decision about the image's acceptability (or otherwise) requires a subjective appraisal, and apparently reconstructed 4K output can look damn good. Let's bring the discussion back to RED. If 4K output can look good on the screen, is there a reason why RED has decided not to offer a 4K output option, or has that decision been made yet?

[Graeme_Nattress]

We haven't said we're not offering a 4k compressed output. There's a lot we've not said, because we're still working things out. We are most certainly offering a 4.5k uncompressed output, similar to the Dalsa.

Graeme

[imgentertainment@mac]

But if you are going to compare these two cameras then I have to ask dose the DALSA offering all the other formats that RED is offering?

And thank Graeme that was a very informative read.

[Graeme_Nattress]

I'll keep trying to be informative. As you know, over the years, I've written articles that de-mystified many aspects of digital video and tried to stomp on some myths too. I think it's great that readers of forums like this can become more educated on the technical aspects of video, and not have to rely on myths and legends to make informed opinions about their productions.

Graeme

[EL_STUPIDO]

Tell me boys,

Does the audience walk out of a film and go "gee that was a great movie, but pity about the 4:4:2 colour space." "yeah, they shoulda totally gone with 4:4:4..."?... in a bell curve: no.

Graeme is right when he stressed about the right lighting... and the right story and even DVX100 miniDV footage can inspire.

The techy side is good to know I guess, and that's what led me to learning about this camera, but once you're on the set, if you're a director it's about actors playing off actors and creating a scene.

Heck, you could have 5:5:5 colour space and if the performance is average, ur stuffed.

El Stupido

[Barry_Green]

Yes, but that argument has little relevance in a camera forum.

We could talk about how the performances would look great on Super 8, too, if it was Al Pacino and Dustin Hoffman.

But that doesn't stop us from using 35mm whenever possible, right? Who in their right mind would waste an opportunity to shoot Pacino and Hoffman, by using Super 8?

The tools are constantly evolving because better is, well, better.

RED makes things better than they were. The HVX makes things better than they were. The DVX made things better than they were.

Discarding the tool as a point of discussion is a rather counterproductive angle for someone to take on a camera-discussion board. It is true that no camera has made a lousy film into a good one, but it is also true that those behind the camera, with the skills, will choose the best tools available to them whenever they can.

Frankly if there are beginners in the RED forum who don't know how to light and who don't know how to get their actors to perform, then they're in the wrong place! A $40,000 camera system is not the right place for someone to be experimenting. This is a serious tool for those who know how to use it.

[EL_STUPIDO]

If the 8mm look suited the scene. Yeah man I'd shoot Pacino on it. Or though maybe I'd treat it that way.

This forum is dragged down by the 4:4:Bore talk. It's like arguing who's better: Kirk or Picard. Who cares?

When's that new form factor being released? There's a seminar soon or something isn't there.

There should be room for wider discussion of the craft in a camera forum like this. You are missing the point of it all.


El Stupido

[EL_STUPIDO]

Or... if you are going to talk amongst yourselves in an exclusive little clique... tell us Cretins w.t.f a Bayer pattern is.

Sounds like a sharp gun used in the French Revolution.

be inclusive, not exclusive... Barry... or whatever your real name is.

El Stupido.

[Barry_Green]

For the love of pete... I'm not missing the point at all. This is the CAMERA forum. Where we discuss CAMERAS.

If you want to discuss the larger picture, why not go to the FILM DISCUSSION section? Or the LIGHTING section? Or the CINEMATOGRAPHY section? Or the SCREENWRITING section?

We're quite inclusive here. We also like to keep some organization.

Sorry, but the whole "it's just a tool" platitude is really, really old. It's obviously not just a tool, it's the whole point the community is here.

Those who don't want to discuss the technical aspects should probably not spend time in the technical sections of the forum, right? We have technical sections for those who want to get as technical as they can. We have general sections for those who don't want to get bogged down in technicals. It's all here.

As for the rest of your post -- I'm really trying to understand here -- your screenname is "el stupido" but you're questioning what my real name is? Is that sarcasm? My screen name is my birth name.

[Graeme_Nattress]

tell us Cretins w.t.f a Bayer pattern is.
El Stupido.

http://en.wikipedia.org/wiki/Bayer_filter

No need for the language.

Graeme

[donatello]

super 8, 16, 4:1:1 , HD , 2k, 4k ,....

the STANDARD that ALL is measured against is 35mm ..
35mm is what 97% of those posting here would prefer to shoot but the COST is out of range ... 35mm lens with 35mm Dof ... 35mm is the WORLD standard .... 35mm is equal to 4:4:4 , 2k-4k or any format- color space when you have it telecined/scanned ... RED will do similar type formats/color space = the user has a choice to shoot in whatever format suits their project and budget and/or use them all in same project all with ONE camera ... for project 1 shoot 720p .. project 2 2k windowed .. project 3 super 16 2k, project 4 1080i etc - also your choice 35mm DoF or 2/3/16 DoF ... 35mm sensor for X $$$

you'll be able BUY or you will be able to RENT or maybe you'll be able to borrow from friend? ... or you go with another camera .. freedom of choice !

[MovieSwede]

what we most of all like would be a digital camera with 35mm look.

Most of us not only uses digital because its cheap, but for its superb workflow.

[Robert Sanders]

Workflow baby! That's what it's all about. Throw in 35mm style sensors and 35mm DOF and I'm a very very happy guy.

[EL_STUPIDO]

It was irony. El Stupido is my brithname too. My parents are cruel spanish communists who locked me in a basement with only a Hamster as my friend till I was 10.

Graeme thanks for that link. I understand more now.

I don't think I was every saying 'it's just a tool'. Obviously I wouldn't be here if I was thinking that...

But your rant on approp forums makes sense. Fair enuff.

I used the percentage of audience awareness anaolgy to start it off. And I come back to that, Subjectivly & Objectivly... how much difference can you really see between 4:4:4 & 4:4:2???

That's not off topic is it????????????????????????????????

El Stupido

[Graeme_Nattress]

Well, in the final image, you probably can't. You can tell in the source footage if you apply lots of effects though that add too much saturation or too much colour correction though. But, because of the way the RAW is converted to RGB, I don't think you'll see any nasties.