Problem Resizing XL2 Footage

[Steve Marshall]

Hello,

I shot some studio footage with a new XL2 and I need to downscale the video to accomodate a graphical border for the final production (approx 18%). The footage is designated for DVD. When I rescale the footage in my NLE (Sony Vegas 5.0) the MPEG-2 render doesn't look crisp and clean on TV playback.

I' ve been trying to resize the footage to my specs outside of Vegas in a program called Virtualdub. So far I don't see any major improvements in the rendered clips.

I shot 16:9/ 60i. I tweaked my custom preset to impart a vivid image. The footage looks clean right out of the camera, but loses something after the rescale and MPEG-2 compression. The best way to describe the defect is a slight overall graininess or artifacting in skin tones.

Does anyone have a suggestion or experience with this issue and the XL2?

Thanks in advance!

Steve

[PDR]

did you do an interlaced resize in vdub (checkmark the box) ?

what format did you export from vdub? or did you frameserve? how did that look at that stage before the mpeg2 encoder?

what mpeg2 encoder did you use? what bitrate was used?

[Steve Marshall]

Hi PDR,

did you do an interlaced resize in vdub (checkmark the box) ?

Yes

what format did you export from vdub? or did you frameserve? how did that look at that stage before the mpeg2 encoder?

No frameserve. Tried uncompressed and then Logarith. It looked brighter in the timeline than the original DV footage even though only the resize filter was applied. After rendering it looked almost identical to the original footage on PC playback.

what mpeg2 encoder did you use? what bitrate was used?

For now Main Concept plug-in in V5 :CBR 8,000,000 - 10 bit Also tried CCE - not yet fully adept with it however.

[PDR]

No frameserve. Tried uncompressed and then Logarith. It looked brighter in the timeline than the original DV footage even though only the resize filter was applied. After rendering it looked almost identical to the original footage on PC playback.

It's a bit complicated, and if you don't care for the details, skip to the 4th paragraph

Whenever you use filters in vdub, it will convert your DV YUY2 4:1:1 to RGB but clamp 16-235 (so you may be losing highlights) . You can get around this by using avisynth scripts and converttorgb(matrix="pc.601") to access the full range. When you export , it will be in RGB full range (assuming you do no other filtering), so you may have to bring it back to legal range in vegas. Another method to avoid the colorspace conversion is to stay in the same colorspace and do an interlaced resize in avisynth. If you want farther instructions, let me know.

Vegas handles different input formats differently. You can read this link below for more info, but there is a levels shift when lagarith is decoded by vegas. You shouldn't get a levels shift if using uncompressed and 8-bit mode in vegas.
http://www.glennchan.info/articles/v...or/v8color.htm

The fact that the footage looked similar in quality (other than the levels), suggests it's the quality of the mpeg2 encoder, or processing in vegas that is the culprit

what mpeg2 encoder did you use? what bitrate was used?
For now Main Concept plug-in in V5 :CBR 8,000,000 - 10 bit Also tried CCE - not yet fully adept with it however.

Sounds like you're using an appropriate bitrate for encoding. CCE is a great encoder when used properly, but it I'm not sure about the MPEG2 quality from that old Mainconcept version... Another great encoder is HCEnc, but it only accepts avs (avisynth) scripts.

What kind of artifacting are you seeing in the skin tones?

If you export uncompressed from vegas, how is the quality => this way you can narrow down if it's something else you're doing in vegas, vs. the MPEG2 encoder

[Steve Marshall]

PDR,

Thanks for all of this detailed info - you're a helluva guy for taking so much of your time to assist me. Sincere thanks my friend!

Although I'm no video newbie, this is the first serious project I’m undertaking for DVD distribution - my work until now has been all PC designated (i.e. interactive CD-ROM, Web-sites, and Digital Signage); these MPEG-2/TV display issues are all new to me. That said, I've never dealt with these encoders, etc.; while I'm reasonably technically savvy, the infinite variables I appear to be facing seem a bit daunting – not to mention the “artifacting”/image degradation I’m seeing only rears its ugly head during DVD playback to TV.

To simplify exactly what I’ve done in V5 so far is this: import captured DV files into the timeline>rescale video approximately 16% (606X404)>brightness adjustment 003/contrast adjustment 015/contrast center 021>saturation to boost midtones (V5 template)>sharpening 0.250 (necessary after rescaling the approximate 16% in V5). I have added a .tga full screen overlay with alpha (“straight unmated”) to a video channel above the rescaled video to provide the bottom/right graphical/info border – that’s it!

May I first ask if you think the 16% resizing/rescaling of the video would cause a discernable loss of image quality? I know the XL2 is a good quality camera and I expect it to look a hell of a lot cleaner than it does in my test clips on DVD! I shot in a decently grid lighted cyclorama studio and tweaked all of my camera settings including a specifically modified custom preset and proper white balance, etc.

To answer some of your questions:

“What kind of artifacting are you seeing in the skin tones?

I can only describe it as a graininess or noisy – the entire video frame suffers as well, though not as prominently; it’s not crisp, a little noisy and not smooth looking.

If you export uncompressed from vegas, how is the quality => this way you can narrow down if it's something else you're doing in vegas, vs. the MPEG2 encoder

The problem is that the degradation isn’t apparent on Pc playback – I am unable to playback .avi files on the TV.

Thanks again for helping me out; I look forward to your suggestions. I have no choice but to figure this out and make this work; I cannot reshoot this thing!!

Best,
Steve

[PDR]

May I first ask if you think the 16% resizing/rescaling of the video would cause a discernable loss of image quality?

It will, but not to the extent that you are implying.

A non interlace aware scaling method will get you artifacts, as the consecutive fields will be treated as frames and you will get messy results

The more you color correct and tweak, the more prone you are to noise and other artifacts, because you are shooting 8-bit video. You can cause artifacts to show up , the more you push. You can reduce this effect somewhat by using 32-bit mode (it interpolates and uses higher precision for intermediate calculations), and also by using a 10-bit intermediate format such as cineform. It's tough to say from your description alone if this is happening

The problem is that the degradation isn’t apparent on Pc playback

What about the MPEG2 export, before you author? i.e. view the MPEG2 stream natively or burned to a DVD on PC playback? Is the grain there? If it is grainy and less crisp, then that highly suggests the MPEG2 encoder and compression issues.

You vegas version is quite old, I'm not sure if there are any problems with that version (I didn't start until using vegas 7) , or the dated encoder

If it's none of these, I'm going to ask to look at a sample of your source DV-AVI, and corresponding sample of your export .m2v elemenatry MPEG2 stream

[Steve Marshall]

“What about the MPEG2 export, before you author? i.e. view the MPEG2 stream natively or burned to a DVD on PC playback? Is the grain there? If it is grainy and less crisp, then that highly suggests the MPEG2 encoder and compression issues.”

Not sure what you mean by “burned to a DVD on PC playback” – should I simply burn the encoded MPEG-2 without using the authoring software (DVD Architect 2.0) to a DVD and playback on my PC? Is that different than opening the encoded MPEG-2 directly from my hard drive?

When I play the encoded MPEG-2 (Main Concept) on my PC in Win Media Player from my H/D it looks ok other than some occasional motion interlacing artifacts which I believe are normal for PC playback. My renders for PC designation are always progressive; as such the interlacing is not as prevalent.

I’m also unclear about your 32 bit mode reference; are you referring to the camera’s user bit configuration or post environment?

Steve

[PDR]

Not sure what you mean by “burned to a DVD on PC playback” – should I simply burn the encoded MPEG-2 without using the authoring software (DVD Architect 2.0) to a DVD and playback on my PC? Is that different than opening the encoded MPEG-2 directly from my hard drive?

Yes, you can open the MPEG2 stream directly on your hard drive.

Please note this may look differently that what you see on a DVD player with regular TV setup. PC monitors use different levels (0-255) than TV levels (16-235) . So it will depend on a variety of factors, what software, how you have your setup, etc...

The basic thing you want to check is do you see the noise in the elementary stream on PC playback

When I play the encoded MPEG-2 (Main Concept) on my PC in Win Media Player from my H/D it looks ok other than some occasional motion interlacing artifacts which I believe are normal for PC playback. My renders for PC designation are always progressive; as such the interlacing is not as prevalent.

If you use dedicated DVD playback software it will bob-deinterlace. In VLC you can activate deinterlace=>bob as well

I’m also unclear about your 32 bit mode reference; are you referring to the camera’s user bit configuration or post environment?

This is a vegas project setting. It might not be available in older versions.


Another thing to check is if architect is allowing pass through of streams. If it re-encodes, you have another generation loss, and lower quality. You should also test the DVDA export on PC playback (e..g mount a disc image, or even play a physical disc with DVD playback software like cyberlink or windvd, then compare it to your regular TV/DVD setup

[Steve Marshall]

PDR,

This thing's getting way too convoluted for me already. Would you be able to give a quick look at a sample mpeg-2 clip and perhaps see what the heck's going on?

The video is for a product which is patent pending (we're in the final stages of the process – the patent is soon to be granted) and I don't want to put a public link to the mpeg-2. If you are willing to take a look I can send you a pm with the mpeg-2 link.

BTW... the mpeg-2 looks horrendous in VLC - way oversaturated and dark for some reason. WMP looks virtually identical to DVD to TV playback saturation/level wise – only the artifacting is not as noticeable.

I went ahead and sent a link - hope it's ok!!

Thanks,
Steve