Codec test: DVCPRO-HD vs. HDV

B

Barry_Green

Moderator
After seeing Eleventy's excellent test demonstrating DVCPRO50 vs. DV 4:1:1 and DV 4:2:0, I decided to see if it could be recreated for HDV and DVCPRO-HD.

Unfortunately it's a lot more complicated, due to the different frame size and the different performance HDV delivers based on whether there's a lot of change in a frame, or little change in the frame. In the end, I decided to test it both ways: best-case and worst-case.

Also, Eleventy's picture is ideal for testing color resolution, but not much else (being, as it is, simply circles of color). Regardless, it's what we had, so we used it.

DISCLAIMER: this is *not* a camera comparison! This is exploring the codecs. The results of this can in no way be used as authoritative proof that an HVX would be superior to an HD100, or that a Z1 would be superior to an HVX, or any other type of conclusion. There are many things that go into making an image, above and beyond the codec. For example, you can shoot regular 4:1:1 DV on a $25,000 SDX900 camera, and you can shoot the exact-same format on a $299 Sharp ViewCam. Does anyone doubt that the results would be night-and-day different? Indeed they would.

So you cannot extrapolate forward and say that a camera using one of these codecs will perform better than a camera using the other codec. That would be an unsubstantiated, and indefensible, leap to a conclusion. The cameras will be compared as cameras when they come out. No comparison of cameras can be made at this time, since neither the HD100 nor the HVX even exist yet.

So, let's dispense with that and get on with the test.

I used PhotoShop to put Eleventy's "Origineel.jpg" picture within a native frame size (960x720, 1280x720, 1440x1080, 1280x1080) frame. I used Sony Vegas to make an HDV-compatible project, and put a native-resolution version of "Origineel" on the timeline. I then output that to an HDV-compatible MPEG-2 transport stream. Then I re-imported that transport stream, and took a snapshot from the timeline (using Best/Full), and pasted that snapshot into a new PhotoShop document. I then zoomed in to 300% and made a composite of all the frames for each resolution.

I enlisted the help of CJ Roy at Arkham Films to make comparable DVCPRO-HD outputs of "Origineel", since Vegas doesn't support DVCPRO-HD. He made the exports and sent me the files back (following the same procedure I outlined above).

For DVCPRO-HD, each and every frame is compressed distinctly and discretely. It doesn't matter what any other frame does, every frame in DVCPRO-HD is compressed exactly the same. Therefore, one frame serves as both a "best case" and "worst case".

For HDV, the same is not true. Depending on what else is happening in your video, HDV will compress an individual frame at varying levels of compression. The more motion, the more change from frame to frame, the more heavily compressed a frame may be. I decided to test the best- and worst-case scenarios for HDV.

For best-case, I used only the "Origineel" file, stretching it out to over 1/2 second on the timeline, and outputting that timeline. This should represent the absolute optimal compression HDV is capable of.

For worst-case, I pushed HDV as far as I possibly could. For 720p I used six different frames from six different projects, each as extremely different from each other as possible, to stress the codec as much as possible. For 1080i I used 15 different frames. I then put the Origineel frame into that mishmash, and extracted the most-compressed version. For 720p, that turned out to be frame #2 in the GOP, for 1080i it was frame #14 in the GOP.

Note: there is no recompression going on. No multiple generations. These pictures are demonstrating one, and only one, compression per frame. The original file was put on the timeline, surrounded by itself or by other frames, and exported once. The results were extracted and displayed here.

Here are the files. Again, it's an extraction blown up to 300% size to exaggerate any artifacts. They're saved as .png files, which are lossless compression, so no new compression artifacts were introduced in this testing. Please right-click and save-as on your computer, and use a photo-editing program such as PhotoShop to view (in native resolution).

720P comparison: http://www.icexpo.com/dvx100/720vs720.png

For 720p, I think the color resolution advantage of 4:2:2 DVCPRO-HD is clearly evident as compared to the blockiness of color resolution in 4:2:0 HDV. However, the HDV, in "best case", seems to show less mosquito noise than DVCPRO-HD. Overall, 720p HDV's "best case" looks quite competitive to DVCPRO-HD; the cleaner mosquito noise may cancel out the color resolution advantage. However, when you get to "worst case", it's a no-brainer. If the average HDV image is anywhere between best-case and worst-case, DVCPRO-HD should have a clear advantage at all times.


1080i comparison: http://www.icexpo.com/dvx100/1080vs1080.PNG

The pictures speak for themselves on this one.
 
Barry, that's great!
Thanks for taking the time to do that.

Would it be alright if I posted your links and text over at "the other place", to add to Eleventy's original test?

Or, if you have the time...feel free to post it over there.
These comparisons are great to see.
 
Luis -- post away if you want. You're a forum leader there, so I'm sure you have more leeway. I don't want to run afoul of Ron's policy of having unique content per site.
 
Thanks Barry - that's a very interesting comparison. Indeed, you might want to extend it further by extracting out the Y, Cb and Cr channels seperately and showing them too, as that will highlight the differences in chroma sampling, and compression.

Of course, it's quite unfair to HDV to show it at it's worst, as that's very hard to achieve in real life shooting, so it would also be good to show a one or two off top compressions to act as a more realistic to real life representation.

Graeme
 
Well, that's the thing -- how to show it at an average? I couldn't think of a way to get this picture to show in average conditions. I could show it at its very best, with a still shot, but that seemed unfair -- nobody's ever going to have the luxury of having a still shot, so the performance we're seeing in the "best" case is just as unrealistic as the "worst" case. And I couldn't think of a way to show it at "average", based on this shot. So I had to figure a way to show it at its best and worst, and let it fall somewhere in the middle.

Can you think of a way to show a still shot like this, but have it be represented in an "average" context?

I was thinking that for a different way of testing, of trying to get ahold of an uncompressed high-resolution computer graphics animation sequence... that would be raw resolution that hadn't been processed by any camera; if I can get it rendered at native frame sizes, it might make for a more representative test.
 
I think an animation would be a good idea. Or even dissolve a still image to black and look through that for good frames to pick from.

Graeme
 
After finding OneRiver's codec testing page, I decided to re-run the codec tests using their picture.

The original picture is at:
http://codecs.onerivermedia.com/enc...eWhite=black&imageWidth=1920&imageHeight=1080

In this round of tests, I tried to take into account HDV's temporal variable resolution and demonstrate a best-case, worst-case, and a new "average" case.

The best case is a still photo for the entire group of pictures. This represents an unrealistically mild level of compression, as you'll likely never encounter completely static footage. It is possible, but unlikely.

The worst case is a completely bit-starved representation, using extreme frame differences to cause the codec to push itself as far as possible. It is probably equally unlikely that you will ever encounter situations that drastic. It is possible that you will, but unlikely.

The "average" case was achieved by doing a simple pan/zoom across the picture. I believe it is generous to HDV, because nothing within the shot is moving; instead it's a slight pan/push across a still shot. The frame that I extracted is the raw un-modified shot; for the rest of the GOP the image was panned maybe 5%, with a maybe 5% pull-out.

This picture that OneRiver uses is very codec-stressful. Even DVCPRO-HD shows significant mosquito noise around several areas. The "picture" area of the image (the ferrari) renders fairly believably under both DVCPRO-HD and HDV, and is probably a more representative "real world" test.

I'll be doing more tests with computer animation, unless someone has some uncompressed high-def footage straight off a Sony F950 they want to send for testing! :)

www.icexpo.com/dvx100/OneRiver-1080-extraction-1.PNG

www.icexpo.com/dvx100/OneRiver-1080-extraction-2.PNG

www.icexpo.com/dvx100/OneRiver-1080-extraction-3.PNG
 
Hey Barry, what's with the wonky line behaviour in the 1080 HDV samples? It looks like some odd combination of misplaced fields. The 720p tests don't show it, because presumably they've been compressed in a progressive stream... but I wonder if there's something wrong with the encoder. Any thoughts?

It doesn't surprise me that the HDV is losing out in the worst cases... but I would have thought the 1080i versions would show similar results as the 720p, and certainly that the best case 1080i would win on a luma test on account of the higher resolution.

Also, how are you handling the pixel aspects? I was a little unsure as to how the original authors dealt with them as well.

-Spiff
 
I'm not sure what's with the "wonky" lines -- presumably it's something to do with 4:2:0 in an interlaced compression system. 1080 in HDV is always interlaced; in 720 it's always progressive, and the 720 showed cleaner non-wonky lines. But, I'm not sure... that's just the results I got.

As for pixel aspect, I gave some thought to that, and I decided that the way to do it would be the way one would do it in real life -- all 1280x1080, or all 1440x1080. In HDV editing you don't set your project to 1920x1080 and then stretch/squish, you set it to 1440x1080 and that's what you use. You would create graphics in 1440, import titles in 1440, etc.

Same with FCP-HD and DVCPRO-HD. When you run a project in FCP, it doesn't set it for 1920x1080, it uses the native codec resolution of 1280x1080. You'll never have a stretch-up/squish-down process, unless you deliberately combined different-sized elements. But for native footage, you'd never do that.

So, I decided to make a 1280x1080 extraction of the original frame, and a 1440x1080 extraction, and used the native resolution version for each codec. That way no other elements came into play (no PhotoShop resizing, etc) -- just native detail.
 
Yes, I think it's issues with 4:2:0 interlaced. 4:1:1 which we're used to with NTSC DV might be thought of as bad, but 4:2:0 interlaced is by far the worst chroma sampling and is tricky to deal with.

Graeme
 
Hey Barry, could you upload some of the 1080 comparisons without the 200% blowup? I'm trying to look at the colour channels individually, but when scaling down the pixel values I'm getting some artifacts from sub-pixel shifts and anti-aliasing. I want to look at each channel of the 4:2:0 and check the effect of deinterlacing etc.

-Steve
 
I've got another question:

Were the DVCPRO-HD 1080 renders flagged as 1080p or 1080i? The wonky HDV artifacts are a combination of 4:2:0 and interlace... but I don't know how much is 4:2:0 and how much is interlace. As an interlaced image spatially compresses each field independently, this could have a strong effect on the perfomance on a progressive frame such as this one.

-Steve
 
CJ Roy at Arkham Films did the DVCPRO-HD render, I didn't. I don't know how he rendered the file out. It was rendered out of FCP-HD.

For HDV, I used Sony Vegas 5.0 with the HDV presets. It uses MainConcept's MPEG-2 encoder, and the preset makes a transport-stream that's will play in an HDV camera.
 
Barry_Green said:
CJ Roy at Arkham Films did the DVCPRO-HD render, I didn't. I don't know how he rendered the file out. It was rendered out of FCP-HD..
Click to expand...

Sorry I haven't gotten to this sooner, there's been a lot of work on my desk as of late.

The codec was rendered 23.98.

I can do a 60i version when I get back from LA next week.

Thanks.

-CJ
www.arkhamfilms.com
 
You must log in or register to reply here.
Back
Top