PDA

View Full Version : Understanding Color Sampling



Jarred Land
12-02-2006, 01:47 PM
Click Here to read the article (http://www.dvxuser.com/articles/colorspace/)

Brandon Rice
12-02-2006, 01:57 PM
WoW! I just read through that! It has totally opened my eyes to what all of that stuff means! Thanks!

Jarred Land
12-02-2006, 01:59 PM
yeah Barry has a good way of explaining things :)

Kholi
12-02-2006, 02:07 PM
And he recommended a Chroma smoother! Great Article.

TimurCivan
12-02-2006, 05:30 PM
enlightening.

ellegrand
12-02-2006, 05:47 PM
Excellent! As always.

TimurCivan
12-02-2006, 05:51 PM
is chroma smootherr a chroma blur?

Chris Messineo
12-02-2006, 06:08 PM
Simply fantastic article. Color sampling was one of those technical things I always assumed I would just never fully understand. Thanks for writing an article so even people like me could get it.

Chris

Graeme_Nattress
12-02-2006, 06:41 PM
Barry - Rods and Cones. Rods are your dark vision. They don't really play a part in normal daylight vision as they're saturated and don't add to the visual picture until it gets really dark. Cones come in three kinds, L, M and S, for receptive to long, medium and short wavelengths. They give us our colour and brightness vision under normal viewing. In the fovea, or central section of the eye, we have only cones present - no rods. There is debate on the L:M:S cone ratios, but is thought to be about 6:3:1 Our black and white "luma" vision comes mostly from both the L and M cones. So we do end up with lower chroma resolution, but it's not based on a rod / cone ratio, but on a L:M:S cone ratio.

Gosh, you can tell I've been reading up on way too much colour science stuff recently!

The example are great Barry - must have taken a while to figure out all the chroma averages. However, any correctly engineered video camera can't do such an abrupt 4:4:4 luma/chroma transition between neighbouring pixels due to the optical low pass filtering etc. Interlaced formats can also contribute to chroma problems, especially with 4:2:0, but they also, by their very nature, have lower vertical chroma and luma resolution anyway, which can make 4:2:0 a better choice in some ways, especially in the PAL world of SD broadcast. Of course, this is rapidly becoming less important.


And thanks for the recommend - The filters are in Film Effects and G Chroma Sharpen uses quite a clever luma adaptive chroma fixing method.

Barry_Green
12-02-2006, 08:01 PM
Well, I'll admit my high school biology was a long time ago... :)

The color sampling wasn't created by a camera, it was done by the codecs. And yes, interlaced chroma on 4:2:0 can be most hideous; the article was meant to be introductory, so I kind of sidestepped the whole interlaced chroma nightmare...

Graeme_Nattress
12-03-2006, 06:34 AM
Well, I'wouldn't have known about cones and rods either until I recently had to swot up on colour stuff. :-)But yes, article meets the goal of being an intro to chroma sampling and gets everyone thinking of the nasties it can do. It's funny though, that if the codec doesnt' average, but just "picks" one colour from the 4 in 4:1:1, you could do chroma reconstruction to produce a better result than you'd get from the averaging. Also, for the most part, when you're up at a decent bit rate, it makes more sense to just compress the chroma more heavily rather than sub-sample it.....

Graeme

Capt Quirk
12-03-2006, 07:51 AM
Barry- What font did you use in the banner? Oh, and it was a well explained article, very easy to understand :)

Jarred Land
12-03-2006, 08:21 AM
I make the banners Capt... that was some photoshop gradients with a pixelation.

nice point Graeme about compressing chroma.. im surprised more people dont do it.

Capt Quirk
12-03-2006, 08:40 AM
Sorry for giving credit to the wrong person Jarrod. So... what font was that?

Jay Rodriguez
12-04-2006, 05:54 AM
nice write up!!!!!! cleared some things up for me but also opened up a couple of questions too.

John Godden
12-04-2006, 11:18 AM
Click Here to read the article (http://www.dvxuser.com/articles/colorspace/)

Jarred

Fabulous article and big thanks also for the link to Poynton's site. :thumbup:

Regards
JohnG

Alex DePew
12-04-2006, 11:39 PM
Great article. Thanks Barry.

SplitVision
12-05-2006, 02:31 PM
wow, that really cleared so much up for me!

Ted Arabian
12-05-2006, 06:15 PM
WoW! That was amazing! I've had people try to explain this to me before and my eyes just rolled in my head! That was SO CLEAR! Thank you!

So.... does that mean that there will be no "averaging" of the pixels in 4:4:4 (as in the Red Camera)?

BTW, I LOVE that banner!!! Very nice!

Thank you, Gentlemen! Well done!

-Ted

Barry_Green
12-05-2006, 06:53 PM
So.... does that mean that there will be no "averaging" of the pixles in 4:4:4 (as in the Red Camera)?
If it's recording 4:4:4, then correct, there will be no averaging of the color values.

Jason Ramsey
12-05-2006, 07:04 PM
This is weird. For the second time in as many days, I see that Barry Green has posted in one of the article threads. I click on that article thread, but his post is nowhere to be found. Does anyone see a Barry Green post right above mine and right below Ted's? I don't. If I search for posts by barry green, I can see it.

Weird,
Jason

Noel Evans
12-07-2006, 04:38 AM
Seems theres confusion over 4:2:0, insofar as the 0 isnt actually 0. Its actually the integer used for 2:1 compression. It does not mean 0 as in nothing.

The XDcam uses 4:2:0 and its a number one seller. So, a lot of HD your seeing comes from this camera. But can you tell the difference on your TV? Weather it be a 42 inch plasma like mine or larger? Un bloody likely. Consider that the US (once again per BG) is mostly still broadcasting SD at 4:3, you can just about achieve 4:4:4 on a downrezzed HD aquisition.

In Japan HD has almost become prevalent (cars here are turned over every year). And the most common output is mpeg2. A 4:2:0 codec. There has been some shifts in decoders in recent years to improve quality. I think its important to understand the rear end so to speak. Japan is at the leading edge in this area......... I submit this article from NHK arguably the most technologically advanced broadcas network in the world : http://www.ebu.ch/en/technical/trev/trev_304-mpeg2.pdf

After reading that do things appear more gray than black and white? Well thats the HDTV world where 4:2:2 does not equal 8 and 4:2:0 does not equal 6.

griffin
12-16-2006, 05:59 PM
Great explanation. Thanks Barry.

My professor in college referred to color sampling as going to a buffet. In a strange way, it made sense.

Luzer
12-17-2006, 08:44 AM
Great article! Finally these numbers make sense to me. And also, the knowledge of colour sampling can be translated into managing your lighting and colour better for your productions.

glenn chan
12-25-2006, 04:00 PM
It's funny though, that if the codec doesnt' average, but just "picks" one colour from the 4 in 4:1:1, you could do chroma reconstruction to produce a better result than you'd get from the averaging.
Hey Graeme, in my tests of that approach it has low RMSE (average error; root mean square error) but visually doesn't look that great. It has inherent aliasing problems, so things like colored lines can lose and gain back their color (in the same image). The decoder for that is also relatively complex, so it won't perform as fast / will cost more. Of course, you've seen my results so you already know this.

2- Barry:
Can I be picky here? :D The diagrams might be slightly wrong compared to actual implemention.

The diagrams you show are what you (well, I) intuitively would guess about the schemes. However, there are actually *multiple* 4:2:0 schemes. This is because the 4:2:0 diagrammed (in your article) has problems with interlacing, and the other 4:2:0 schemes try to solve that.

If you notice in interlacing, the fields are taken from two different points in time. However, the chroma is averaged from two different points in time. This time mismatch means that the chroma is inappropriate if you have motion going on.

To fix this, you can break the image up into two fields, and 4:2:0 sample each field. This way, the chroma samples are taken from the right slice of time, so you won't have problems with motion. However, you introduce new problems. Each chroma sample, effectively, describes the chroma for an area ~2 pixels wide by 4 pixels tall (ignore co-siting for now). This is actually not as good compared to 4:1:1, where the chroma samples describe a ~4 X 1 area (again, ignore co-siting for now).

You also get another problem in that the artifacts look comb-like, instead of a smooth transition.

See
http://www.hometheaterhifi.com/volume_8_2/dvd-benchmark-special-report-chroma-bug-4-2001.html (somewhat related)
And the 4:2:0 follies article by adam wilt in DV magazine is very relevant here, but unfortunately it's not on dv.com anymore.

The following formats use different 4:2:0 variants:
PAL DV (except for DVCPRO PAL, which is 4:1:1)
MPEG2/DVD - DVD uses two schemes, one interlaced sampling and the other progressive
?HDV seems to be the same as in DVD. The Main Concept HDV encoder is consistent with MPEG2/DVD in my testing.


3- The chroma samples tend to be centered over a particular pixel, and describe the chroma over an area. i.e. for 4:1:1, a chroma sample can be taken from the chroma values from 7 different pixels (assuming a simple filtering scheme; some filtering schemes look at a much larger number of pixels).

On a professional VTRs, they seem to apply some chroma interpolation,
so the missing chroma values are created by looking at the chroma samples to the left and right (i.e. TWO chroma samples). Compare and contrast this to what most DV codecs do: reconstruction is based upon a SINGLE chroma sample. Your diagram shows what most DV codecs do. While I don't believe what most DV codecs do is entirely correct (it shifts the chroma horizontally), that chroma shifting makes other problems look less bad. The scheme used in most DV codecs also has good multi-generation performance.


4- If you want to get really hardcore, you could look at how pixel shifting the in camera (i.e. the DVX100 exploys pixel shifting) affects effective color quality. The effects of pixel shifting happen before compression and before chroma subsampling. For ideal 4:4:4 color, you would not employ pixel shifting (although pixel shifting may yields better pictures / MTF).

I have never tested with the DVX100, but my Panasonic GS70 (NTSC) does pixel shifting and employs heavy filtering. Its color resolution is terrible (both vertically and horizontally).

Luzer
12-29-2006, 12:17 PM
So Glenn,

Are there any real world examples of frame grabs or video clips to show the image quality before and after pixel shifting? Perhaps you can share what you have tested with the GS70?

Barry_Green
12-29-2006, 12:34 PM
The diagrams you show are what you (well, I) intuitively would guess about the schemes.
Well, no, they're not guesses. They're actual pixel-for-pixel extractions of what the codec does to the footage. I ran the pixel-based 4:4:4 image through each of the codecs (DV NTSC for 4:1:1, DV PAL for 4:2:0, DV50 NTSC for 4:2:2). So the colors that are generated and demonstrated there are what actually happens.

Yes, interlaced 4:2:0 is a nightmare, which may be why Panasonic chose 4:1:1 for DVCPRO in PAL... and again, the article isn't meant to be all-encompassing, it's meant to provide a basic overview so that the numbers start to make some sense. :)

glenn chan
12-29-2006, 01:37 PM
Hey Luzer,

You can't un-pixel shift the GS70 easily since the CCDs are aligned that way at the factory, and the DSP in the camera is designed for pixel shifting.

I suppose it would be possible to measure luminance and color resolution simultaneously, to show you the relative differences between the two.

If you display the following pattern, it's a good test of resolution + aliasing.
http://glennchan.info/Proofs/forums/double-zone.png

Unfortunately, that test pattern is very difficult for monitors to display correctly (and I haven't figured out how to print it correctly). Resizing it bigger 2X via **nearest neighbour** might help (or zoom in 200%, which does the same thing in Photoshop).

2- Barry:
They're incorrect in the sense that they don't show the correct way of doing things (i.e. they don't show co-siting). Although in the case of the apple NTSC DV codec, your 4:1:1 diagram should be correct.

For the DV50 4:2:2 codec, the correct diagram for that codec would be a little different and would show co-siting (since that's what the DV50 codec does). The chroma samples are positioned over a luma sample, not between them. And in the case of Apple DV50, the sample is taken as an average of:
0.25 times the chroma to the left
0.5 of the chroma it's sitting on
0.25 times the chroma to the right

Of course, that makes things a little harder to explain.

KyleProhaska
01-02-2007, 06:58 AM
Inciteful, thanks for this article.

SamKam
01-05-2007, 10:11 AM
Barry, thanx for this article. Cogently and concisely explained. Very helpful.

SamKam
01-07-2007, 01:09 PM
Thanks Barry. Very well explained and extremely helpful.

poopnoodle
01-29-2007, 06:33 AM
good stuff:happy:

3CCD
02-05-2007, 05:42 PM
Good stuff, I learned quite a bit.

ronik15
02-06-2007, 10:40 AM
Thanks for the article now I understand color sampling.

churchie04
02-06-2007, 01:49 PM
Great article. Just had to throw my thanks in. I understand this stuff so much better now.

KyleProhaska
02-07-2007, 06:52 AM
Excellent article, really gives the black and white answers to what can be done with what sampling and what you miss without the better samples. Good read as always.

- Kyle

tb9
11-11-2007, 02:15 PM
Wow great article- ive noticed this in digital photography too- especially on kit lens

SonicStates
02-15-2008, 02:58 AM
As per! That was very enlightening indeed. Thanks.

Marshallarts
02-19-2008, 09:07 PM
Long time knower, first time understander.

Hope that conveys my thanks for your excellent explanation...!

Goldmond
04-14-2008, 04:08 PM
This is a good intro, looks like I'll be buying some kind of book on color correcting.

ficlead
05-13-2008, 11:14 AM
Great stuff. Thanks!

CG
5:01 Studios

jlabaudio
11-02-2008, 09:16 PM
interesting article... so what are the best camera choices for green screen? what about budget cameras?

thanks,
Josh
http://www.JLabAudio.com

momentfilms
11-03-2008, 10:33 AM
thanks, very interesting!

Luis_
02-22-2009, 02:18 PM
Thank you very much for this article.

Robert Ruffo
01-09-2010, 01:12 PM
I'll be posting a review of the Redrock Me2 soon - Wow, edge-to-edge is way better on our Me2 than this!

J Davis
01-10-2010, 03:55 AM
Hi Barry,
I've always thought that the three numbers separated by colons (eg 4:2:2 ) stood for y Cb Cr (luma, channel blue, channel red) and
that channel green was calculated mathematically by the difference of the other two channels.
And going on my understanding, prores 4:4:4:4 would then be y, Cb, Cr, Cg.
edit: or more likely 4:4:4:4: is y, Cb, Cr, alpha
Is there any truth to that or am I laboring under a misconception?

Thanks

J Davis
writer / director
jdMAX.com

JeganRX
01-12-2010, 07:15 AM
Hi Barry,
I've always thought that the three numbers separated by colons (eg 4:2:2 ) stood for y Cb Cr (luma, channel blue, channel red) and
that channel green was calculated mathematically by the difference of the other two channels.
And going on my understanding, prores 4:4:4:4 would then be y, Cb, Cr, Cg.
edit: or more likely 4:4:4:4: is y, Cb, Cr, alpha
Is there any truth to that or am I laboring under a misconception?

Not quite... the whole YCbCr (component) concept is simply the method of storing RGB as a form of color differences - as you said. However, you can still have 4:4:4 YCbCr (I don't know a format that allows this, but it's theoretically possible). True, this discards some color information, as any color conversion does, but it still maintains the same resolution, assuming it was 4:4:4.

Color Subsampling refers to that ratio you're mentioning, and this is where we often see 4:2:0, 4:1:1, 3:1:1, 4:2:2, etc... It's usually used in conjunction with color difference formats, such as YCbCr or YUV (which are similar systems, though use different values for their RGB conversions if I'm not mistaken), hence the confusion here.

Of course, then we have bit depth, but that's another story!

Final Note: YCbCr is not the same as YPbPr, though in modern times, the terms are used interchangeably. Confusing, isn't it?

J Davis
01-12-2010, 11:01 AM
I just had confirmation from Barry (via PM) that the numbers do indeed refer to the Y (luma), Cb (blue), Cr (red) channels. I didn't hear back from about what the fourth digit in prores4444 represents. My bet is that it is alpha.

Hopefully he will chime in here with the correct info

edit:
I would also be interested in knowing if the nomenclature can only stand for y Cb Cr or whether it can also refer to YUV as written above by RX782

adkimery
01-12-2010, 06:18 PM
The 4rth 4 in ProRes 4:4:4:4 is the for the Alpha.


-A

JeganRX
01-14-2010, 10:03 AM
I just had confirmation from Barry (via PM) that the numbers do indeed refer to the Y (luma), Cb (blue), Cr (red) channels. I didn't hear back from about what the fourth digit in prores4444 represents. My bet is that it is alpha.

Hopefully he will chime in here with the correct info

edit:
I would also be interested in knowing if the nomenclature can only stand for y Cb Cr or whether it can also refer to YUV as written above by RX782
On a strictly technical level, they are two different systems, but again, the only difference being the coefficients used to convert RGB to R/B difference signals; however, yes, the terms are often used interchangeably.

Shiloh Arts
03-03-2010, 12:00 PM
Ok-- so, in Barry's article -- which is great btw -- he mentions that DVCPRO-HD uses 4:2:2 color sampling, but Panasonic markets their HPX 3000 camera as providing 4:2:2 using AVC-Intra codec. Is this true, or should I be using DVCPRO HD for green screen shoots instead?

Barry_Green
03-03-2010, 12:41 PM
AVC-Intra (at 100mbps) and DVCPRO-HD are both 4:2:2. However, AVC-Intra100 is better in pretty much all ways than DVCPRO-HD, so always use it whenever you can.

However, AVC-Intra50 is 4:2:0, so I would definitely use DVCPRO-HD instead of AVC-Intra50.

Cranky
03-03-2010, 01:12 PM
If XDCAM 422 were thrown into the comparison, how would you rank AVC-Intra50, XDCAM 422 and DVCPRO-HD in order of your preference?

Barry_Green
03-03-2010, 04:36 PM
Loaded question. XDCAM 422 is only available as a long-GoP codec on a disc, vs. solid-state intraframe codecs. If Sony would make 422 available on SxS, we'd be talking.

XDCAM 422's pretty good. I'd probably rank it between DVCPRO-HD and AVC-Intra100.

Cranky
03-03-2010, 05:24 PM
Loaded question. XDCAM 422 is only available as a long-GoP codec on a disc, vs. solid-state intraframe codecs. If Sony would make 422 available on SxS, we'd be talking.

XDCAM 422's pretty good. I'd probably rank it between DVCPRO-HD and AVC-Intra100.
I believe what Canon is calling "MPEG-2 HD @ 50 Mbit/s with 4:2:2" pretty much amounts to XDCAM 422, the latter is just shorter to write. Or maybe Canon decided not to license Sony's file structure and have only IMX files. Anyway, when I asked the question I meant this "new codec" from Canon as well.

So, you would put it below AVC-I 100. What about AVC-I 50?

Barry_Green
03-03-2010, 06:26 PM
Of course it's below AVCI-100, it's an 8-bit codec, and long-GoP at that. But I'm not all that thrilled with AVC-I 50. I don't think it looks as good as DVCPRO-HD. Of course, at half the bitrate, it doesn't really have to look as good in order to justify its existence, but no, AVC-I50 isn't as good as DVCPRO-HD and I wouldn't expect it to look as good as XDCAM 422 either.

Shiloh Arts
03-04-2010, 08:42 AM
Thanks Barry!

Last question -- and I'll be on my merry way is it correct to assume that 4:4:4 sampling won't show a visible difference raw over 4:2:2, but only AFTER color correction, or have I completely missed the point?