Wanted: CODEC Cook-off

[RBG]

Thanks Barry Green for your informative and entertaining
"Canon 7D, meet - some peers" which I just read.
http://www.dvxuser.com/V6/showthread.php?t=186911

(At least I clued into your post #167 re "impossible to ship all 3 cameras.")

What I'd also dearly like to see are some real-world comparisons of the image qualities from the myriad compression codecs (as produced by their cameras). I'm talking about the various rates and flavors of AVCHD, MPEG2, AVC-I, XDCAM EX, other MPEG4, the "4 simultaneous DV codecs" that is DVCPro-HD, MJPEG, and so forth.

In my attempt at self-education, I can attest to the fact that you can Google all day (weeks as I have) and find not a thing about the relative IQs of these codecs. Lots on compression rates. Lots on other tech specs. Nothing on what they actually look like on the screen.

Is the "new, advanced, super-efficient high-tech" AVCHD 24Mb/s better than the "old-school, yesterday's-news MPEG2 I frame?" Who knows? Especially if you don't give a hoot about efficiency in today's world of cheap & plentiful memory. What if your more important criteria is image quality and lack of compression artifacts?

For me, nothing says low quality (& amateur) than digitally breaking-up footage. Think MPEG1. Personally I can't even consider sharpness and colour rendition unless, at least, image integrity needs are met. (Maybe I was spoiled by 3/4 inch? ;^)

So needed is a side-by side comparison and/or judgment-call derived from various cameras running difficult scenes like sports action, windy leaves, waterfalls, that sort of thing.

As a benchmark, I would suggest comparing against a very good Blu-ray movie since that is what audiences are required to accept.

As an HDCAM producer, I find the new technologies both mesmerizing and a tad bewildering.

RBG

[RBG]

Ooops. Meant to put this in with Canon 7D. And then wasn't even sure that's where it should go. Bad first post, bad.

RBG

[Barry_Green]

But what you're asking for is nearly impossible to quantify. Trust me, I've organized six-way and three-way camera shootouts before, and the ONLY true, discernible results, are that a) everyone whose favorite camera didn't "win" hates me, and b) they choose to not believe the results either. They think there has to be "bias" or that something wasn't "tested right".

It's a complete no-win. You cannot possibly scientifically evaluate multiple camera, multiple codecs, multiple scenarios, and come up with anything that will hold water against any particular argument anyone would choose to make.

So -- check out my XDCAM EX vs. AVCCAM thread -- that's the only way that I know of, that it can be done, that removes *all* variables, and nobody can possibly argue with the results. Well, no sane person. Anyone disputing the results is easily classifiable as a whining doofus, because in that test, absolutely only one variable existed -- the particular codec.

So -- if you want to know how codecs perform, that (IMO) would be the way to do it -- get an external SDI recorder using your particular codec of choice, and hook it up to a known uncompressed SDI source, and have at it.

Is AVCHD 24mb/s better than old-school MPEG2 I-frame-only? Depends on the bitrate. 24mbps vs. 24mbps? AVCHD will decimate and obliterate MPEG-2, whether it's I-frame or long-GoP, doesn't matter. 24mbps AVCHD vs. 35mbps of long-GoP MPEG-2? No question, AVCHD is better. 50mbps I-frame MPEG-2? I don't know, my GUESS is the AVCHD still wins. 100mbps of MPEG-2? Eh, now you're talking -- probably MPEG-2 wins that one. But at the cost of 4x the file size, so how do you call it a "winner"?

If your primary focus is on image quality and lack of compression artifacts, you don't want *any* long-GoP codec. Your choice at that point is easy: AVC-Intra 100mbps. 10 bits, intraframe, H.264, certified "Gold" by Discovery HD, chosen by the BBC as their codec of choice for archiving their Digital Media Initiative footage, and given a technical achievement award by the Hollywood Post Alliance. If you can go upscale, you also would seriously consider HDD5 or HDCAM SR, although both of them use much more bandwidth and are not available in an affordable camcorder configuration.

[RBG]

Thanks for that Barry.

I got a little hint of the criticism that one can attract in that Canon 7D item of yours.

But all your thoughtful objections aside ;^) how does there need to be any (or much) science to what I am asking?

Three cameras shoot a complex motion scene.

Camera one: Exhibits no discernable digital blocking.
Camera two: Clear blocking here, here and here as illustrated within circles.
Camera three: Even more degradation of scene in the same areas.

Then: "My opinion. YMMV. You look at it & decide for yourself."

Your quick response was very interesting to me, regardless. I had no idea 24Mb/s AVCHD would hold up so well, comparatively speaking. I am completely surprised that 24 Mb/s AVCHD would outperform 35Mb/s I-frame MPEG 2. How can this be? Assuming AVCHD has the hell compressed out of it compared to MPEG 2 which additionally is at a higher bit rate. (But perhaps that extra MPEG2 bit rate is needed just to maintain so much I-frame info?) So lots still to learn here.

You now make it difficult to discern whether a GY-H100 running 35Mb/s XDCAM-EX codec
is inherently better than a 24Mb/s HMC150 - lenses, image detectors, and everything else aside. Which would show less compression artifacting on the same scene? That doesn't have to be a difficult question, I don't think.

Thanks for that codec overview including all the higher-end codecs. (My own curiosity is more limited to the under $5,000 cameras.)

I'll check out that other referenced thread.

RBG

[Barry_Green]

Camera one: Exhibits no discernable digital blocking.
Camera two: Clear blocking here, here and here as illustrated within circles.
Camera three: Even more degradation of scene in the same areas.

Then: "My opinion. YMMV. You look at it & decide for yourself."
I say go right ahead. I know from past experience, absolutely nobody will be satisfied, complaints will rise, people will perform alternate tests designed to make their chosen product look good and post those results, and children will cry and small animals will be kicked. At least, that's what happens every time I do any sort of comparison...


I had no idea 24Mb/s AVCHD would hold up so well, comparatively speaking. I am completely surprised that 24 Mb/s AVCHD would outperform 35Mb/s I-frame MPEG 2. How can this be?
http://www.dvxuser.com/articles/article.php/25

AVCHD @ 21mbps average, proves itself to be better and more resilient than 35mbps of MPEG-2. And that's long-GoP MPEG-2, which is more efficient than intraframe would be.


Assuming AVCHD has the hell compressed out of it compared to MPEG 2 which additionally is at a higher bit rate.
But AVC is a far, far more efficient codec. It's more advanced, and simply better at compression, than MPEG-2 is. Sony and Panasonic issued a joint statement saying that AVC is two to 2.5 times more efficient than MPEG-2. And Sony has separately said that 9 megabits of AVC is visually comparable to 25 megabits of MPEG-2 HDV.

So imagine a car analogy; MPEG-2 is like a car that gets 20 miles to the gallon; AVC gets about 45 miles to the gallon. So give MPEG-2 35 gallons, and give AVC 21 gallons -- which car will drive farther? See, you can't just go comparing bitrates and say "35 > 21" unless you know the rest of the story.

Now, take Sony's third claim, that long-GoP makes a codec about 3x as efficient as intraframe -- they're basically saying that you'd need 100mbps of intraframe MPEG-2 to make images as compact and well-compressed as 35mbps of long-GoP MPEG-2. So by that comparison, 21mbps of AVCHD should stand up quite well to, or be even better-performing than, 100mbps of intraframe MPEG-2.


You now make it difficult to discern whether a GY-H100 running 35Mb/s XDCAM-EX codec is inherently better than a 24Mb/s HMC150 - lenses, image detectors, and everything else aside.
Actually, no, it makes it far easier. A 24mbps HMC150's codec will perform better than a 35mbps XDCAM-EX codec on the GY-HM100. No question, no problem, no doubt. Especially if you factor in that the HM100's image might be slightly sharper, which would tax the codec further than the HMC150's slightly softer picture (assuming, of course, that that's what actually happens; we're not factoring in 1/4 vs. 1/3", etc. so unless I try them side by side, I'm not prepared to say that the HM100 is actually sharper. But if it was, then the codec performance difference would be even greater in favor of the HMC150/AVC combo).


Which would show less compression artifacting on the same scene? That doesn't have to be a difficult question, I don't think.
It isn't a difficult question. The HM100 will show more compression artifacting, unquestionably.

[PerroneFord]

I love these discussions...

It is my view that:

1. DVCProHD runs at 100Mb because it needs to to adequately capture the 1280x1080 signal it has to capture with I-Frames

2. AVCHD in a good implementation can be very very clean. Which is why we see Blu-Ray discs coming from hollywood encoded at about that rate to transcode 2k footage down to 1080p

3. XDCamEX at 35 Mbps exists because it's "good enough" for most things people will point a handycam at, but is clearly bested by the more efficient 24 Mbps AVCHD.

4. I will be doing some side by side tests of AVCHD and XDCam at 50Mbps to see what is cleaner on the timeline.

5. 100 Mpbs XDCam EX (long GOP) is going to beat 24Mbps AVCHD. However 100Mbps I-Frame XDCamEX is another matter entirely. It might beat 24Mbps AVCHD, maybe not. That's a test I'd love to see.

6. A good wavelet codec at the same bitrate will take either of these codecs behind the woodshed... I've done those tests.

[Barry_Green]

4. I will be doing some side by side tests of AVCHD and XDCam at 50Mbps to see what is cleaner on the timeline.
But 50mbps is 4:2:2, so it won't be directly comparable. Well, I mean, it shouldn't be...


5. 100 Mpbs XDCam EX (long GOP) is going to beat 24Mbps AVCHD.
It should; 35mbps was very very close in scenes where the codec was't stressed. But tripling the bandwidth should give it enough to handle the stressed situations. But -- and, as Pee Wee Herman said, everyone has a big but -- if the 100mbps is 4:2:2 instead of 4:2:0, then you just threw an additional monkey wrench in the works, because then it has to encode a lot more data. So -- will it beat 24mbps of AVCHD? For color resolution, easily, but for codec resiliency? Maybe, but maybe not, or at least not as much as if it was still 4:2:0.


However 100Mbps I-Frame XDCamEX is another matter entirely. It might beat 24Mbps AVCHD, maybe not. That's a test I'd love to see.
According to Sony's claim of 3x more efficiency, I think your guess is about right -- they'd probably be very close. I think the 4:2:0/4:2:2 thing might determine it.

[PerroneFord]

Right, the 4:2:0 vs 4:2:2 thing really will make a difference. The thing is, has it become academic at that point? We are marveling over the quality of Blu-Ray at 1080p 20-25Mbps 4:2:0.

I do think it's telling that the Digital Cinema Initiative group has settled on Jpeg2000 at 280Mbps 4:4:2 max for 4k digital display in the theater. I don't know if you've had a change to look at wavelet at 100Mbps+ but I sure have. 280Mbps might as well be lossless. I've done A/B test at 50 Mpbs with 1080p 4:4:2 and it might as well have been uncompressed.

We've come so far since the days of MiniDV...

[Barry_Green]

Wavelet is sweet because it's a more organic type of compression. Have you ever looked into fractal compression? There's absolute engineering genius in it...

If you've looked into AVC, it's just crazy -- they have motion vectors and prediction not just for objects in the frame but for the way things actually move. It knows what a "human" shape is, it knows how to predict when an arm will move and how it will bend at the elbow... I marvel at the compression sytems we have now. I pick on long-GoP not because of the technology, but because of the current implementations. As a technology it's brilliant, I just want to start with a solid intraframe and get better.

Wavelet/JPEG2000 is the stuff too. Wavelet is great because you can do partial decodes and still get the basics of what you're looking for; you can dig into it as deep as you want to get as much quality as you want. I know panasonic's built a wavelet proecessor for JPEG2000 for their D5 HD decks to turn 'em into Hollywood mastering decks for the DCI; I'm thinking if they've already got it... and P2 cards are rated for 1200mbps of data transfer... um, why not give us some JPG2K?

Of course, 280mbps data files... three gigs per minute... that'd be quite the data management situation.

[PerroneFord]

Of course, 280mbps data files... three gigs per minute... that'd be quite the data management situation.

Not really, I am wrangling 220Mb 10-bit DNxHD files on my laptop right now for the film project I'm working on.

I did a codec sample tonight pitting DNx36, DNx175, and J2K at 10% quality (132Mbps) against each other using a difference mask. J2K won handily. And this is why its my archive codec of choice.

[Barry_Green]

What codec? And did you render to them from Vegas?

[PerroneFord]

Cedec was JPeg2000, and it's a Vegas built-in. Best archival codec out there, and DARN few people realize Vegas has a free built-in rival to REDCODE, but without the metadata and flexibility. When I learned that Vegas had wavelet built-in after scurrying tring to get a third party J2K or Dirac codec installed, I wanted to kick myself. I changed my archiving from VC-3 to Jpeg2k immediately.

[RBG]

BG: "But AVC is a far, far more efficient codec. It's more advanced, and simply better at compression, than MPEG-2 is."

When I think codec efficiency, I think first the ability to encode more stretches of redundant or similar 1s & 0s into a smaller space and second, the ability to also predict such encodings. The bottom line being a hell of a lot of digital information necessarily being unintentionally mutated in the process. Yes, that kind of compression would be hightly efficient. And it follows that the more you compress, the more the codec could be considered efficient - and the more it mutates from the original uncompressed. It should be a technologically beautiful, but ultimately ugly, thing, in my naive opinion.

The more you compress, the further you move from the original uncompressed. In that light, so what if AVCHD is 2.5 times more efficient than MPEG2? We consider HDCAM SR and we are thankful that it is only extremely lightly compressed.

BG: "(think of it as miles per gallon in a car; MPEG-2 might get 20 miles per gallon, AVC gets about 45 miles per gallon in comparison)."

Then following this analogy, uncompressed would get 2 miles a gallon. But who wouldn't want uncompressed quality like they'd also want the record low fuel-economy Lamborghini Murcielago?

But of course I am missing something here.

RBG

[PerroneFord]

But of course I am missing something here.

RBG

You are.

It's called a coDEC. The process has two sides. On one side the COmpression. On the other the DECompression. Codec efficiency is indeed the ability of the codec to compress files into a smaller space. Codec quality however is the ability of that codec to return us as closely as possible to the original state.

This is quite easy to do when not much compression is done in the first place. But another matter entirely when using a codec that can compress files down 10:1 or more. This is why codecs like Jpeg2000, Cineform (the higher end stuff), and REDCode are so amazing at what they do. They compress at 8:1 to 20:1 and can return us either exactly, or nearly exactly to what we started with. And they can do it in such a way as to not slow our machines to a crawl when we want to view that compressed file.

This is where AVCHD falls down. It is a very complex codec, and it requires some serious computation to return us back to a viewable file. This is why it's so slow on the timeline. I can play a REDCODE 4K file on the timeline about as smoothly as I can play a 1080p AVCCAM file. And the REDCODE file will retain far more detail.

[RBG]

Geez, Barry. This article is completely great. I doff my cap in your general direction. May I suggest you watch over your shoulder as stuff like that could bring out the inner hitman in certain Vested Interests.

http://www.dvxuser.com/articles/article.php/25 (http://www.dvxuser.com/articles/article.php/25)

More gruel please. ('Course it would be neat to see motion because I am guessing motion might have the ability to mask some of the blockiness(?))

Now: DVCPro HD vs 21Mb/s AVCHD? No contest but, how much better? What other matchups might be interesting?

So I take away from that article that AVCHD is efficient not just because of its straight compression ability but because of the way it uses variable-sized blocks, it can best certain (all?) MPEG2 compressions. I'll have to think about the auto analogy for that one.

RBG

[RBG]

#14 PerroneFord

I understand what you are saying. It is also the reason I could so easily run 4 simultaneous streams of uncompressed SD YUV via my Targa3000 board but only 2 DV.

But my concerns were more related to the quality of the image and less about what I perceived was "efficiency:" the amount of compression involved. I'm beginning from the premise that the more one manipulates a digital image the more one adds digital mistakes and compromises into the digital streams. That is, the more an image is compressed (and uncompressed), the less it is like the original. In the same way the more one transcodes a purely digital image, the more that image will nevertheless still degrade.

The answer obvious to me now is that while the above is true, if one must compress for practical reasons, codecs can have different levels of technical sophistication or "efficiencies." It's not only a matter of degrees of compression (my use of "efficiency").

So a lightly compressing codec still might not look as good as a greatly compressing codec employing a far superior codec algorythm. By Jove, I think the lad has got it.

RBG

[Barry_Green]

Geez, Barry. This article is completely great. I doff my cap in your general direction. May I suggest you watch over your shoulder as stuff like that could bring out the inner hitman in certain Vested Interests.
Five days after I wrote that article, Sony introduced their own 24mbps AVCHD. So the "Vested Interests" shouldn't really complain all that much.

And hey, anyone anywhere is free to replicate my experiment. If they can prove it wrong, go at it. But the problem is -- they can't, because what i demonstrated is exactly what happens. So -- sorry, Vested Interests, but what is, is what is. :D


('Course it would be neat to see motion because I am guessing motion might have the ability to mask some of the blockiness(?))
Nearly impossible request to satisfy. How would you propose to view such -- on a Vimeo side-by-side? The only way something like that could be done, is to encode both motion segments side-by-side in a nearly-uncompressed or actually uncompressed codec, and the file sizes would make such a thing prohibitive.

And yes, motion hides lots of sins.


Now: DVCPro HD vs 21Mb/s AVCHD? No contest but, how much better? What other matchups might be interesting?
Intraframe indestructibility vs. long-GoP, and 4:2:2 vs. 4:2:0, would give DVCPRO-HD a large advantage. AVCHD would have full-raster on its side, however. I wouldn't be surprised if the end result of some substantial comparing ended up showing DVCPRO-HD with a small advantage.

[Barry_Green]

So a lightly compressing codec still might not look as good as a greatly compressing codec employing a far superior codec algorythm. By Jove, I think the lad has got it.
I would say that this is correct. The question is how effectively the compression/decompression cycle results in recreating an accurate representation of the original. A more-efficient codec can accomplish that task in less bandwidth than a less-efficient codec. The ultimate is to take a highly efficient codec and then give it a lot of bandwidth.

[RBG]

Funny, I was literally mid-read of a good review of that Sony camera, the NXCAM, on another similarly-named British site when I realized that I had spelled alorithm incorrectly, causing me to dash back here.

So now it will be the 20X zoom CMOS Sony vs the CCD Panasonic.

RBG

[RBG]

Somebody just shoot me. Algorithm. Algorithm.

It's Al Gore I tell you.

RBG