squeeze or letterbox

[Jack Daniel Stanley]

Cool. I have not seen that one. Moved to HVX a while back so haven't been looking for uprezz options since then much. For ex. I think the PhotoZoom
Pro software I mentioned has been sold or changed names now or something.

 

[Akatastasia]

Hey Jack great post. I have two questions. When you say use FCP to blow up, do you mena do that on export? Is there a special blow setting in compressor? I have never done this before.

2. I am going to be posting a lot of things on the web. Is there a way to letter box, but actually "cut" the black out so that the black strips don't code into the file, thus minimizing the file size for download over the internet. I want to make the files as small as possible and I figure I can save some space by somehow getting rid of the black bars.

Thanks

 

[Jack Daniel Stanley]

you can do it two ways.

1) edit in your 4:3 timeline, then make a new 16X9 sequence and drag the original 4:3 sequence onto the new 16X9 timeline. To get a nested sequence to open in the Viewer you have to drag it to the video window of the viewer or right click it and choose open in viewer. There, in the motion tab, adjust scale and aspect ratio (under distort) as appropriat / as needed. The scale should be about 133 I think, and I forget the distortion / aspect ratio adjustment, but you can just drag it to the left and right until the blue overlay outline meets the edge of the frame on the sides ...
2) In compressor under the geometry tab, there's a new "crop to letterboxed area of source" check box. So check this and this will auto-crop your black bars out. Now your video will look squished. So then in the lower half of the same geometry window choose 16X9 aspect ratio and it will stretch the video between the black bars to be actual 16X9.

Of the above, I would recommend 2 probably with the dual pass setting. For two reasons.
1) compressor can really take its time and do the two pass render and maybe giving better results since its not optimized to render as quicky as FCP ??? just a theory.
2) you have a clean pixel for pixel 4:3 master in your NLE or you uncompressed master version that you output and you can use that and crop or stretch in compressor from a cleaner starting point to go out to whatever you want ... burning a copy to DVD for Mom and Dad and their big screen 4:3 television, great, burning a copy for that Ad Agency with all the 16X9 screens all over their office, no problem, need a copy for the web with the black bars cut off? you've got it.

Which brings up your second question, how to output without black bars for the web. Most of my stuff is letterboxed even further to 2.35:1 (cinemascope) and I throw all of those black bars away for the web. Look at "Unawakening" in my signature to see a web video compressed this way, or any of the videos on my website.

Basically you follow the steps for #2 above. But you want to make sure your pixels are square for the web so the video is not squeezed. If compressor gives you any problems on achieving this (don't remember how easy the setting is offhand, maybe it is auto now) ... but if it isn't working or you can't figure out the settings in compressor then, after you have output your cropped, but now squeezed video, you can just open it in QT pro, "Show Movie Properties"/"Visual Track" and change the shape of the video by entering a new length and width and click "save as" and you are done and ready to upload.

 

[Joshua Provost]

Jack, I couldn't disagree with you more about SQUEEZE mode. You have a number of factual errors that lead to wrong conclusions.

Its 720X360 blown up to 720X480. But the camera has not recorded 720X480 at 1.2, it's only recorded the center 360 or so, which gets stretched vertically (so the name Squeeze is a technical misnomer, but its called that because it looks squeezed)

The chipset for the DVX is native .9 pixel aspect or 4:3, to get 1.2 and 16X9 it fudges the image. While letterboxing does not work against the native chip set. It just covers part of the image.

The DVX CCD is not 720x480. It is 720x576. The same CCD is used in both NTSC and PAL models. Check the specs, the CCD is 410k pixels. That's 720x576.

Therefore, the quality of SQUEEZE mode depends on which model you are using. In NTSC-land, SQUEEZE mode is very good. It's about 430 true lines of resultion expanded to 480 (not 360 to 480 as you say). In PAL-land, your arguments may have some merit.

Also, consider that the SQUEEZE uprez occurs in-camera prior to color channel subsampling to 4:1:1 and prior to DV 5:1 DCT compression. It's a clean full image off the DSP that is uprezzed prior to all of that image-damaging processing required to get it to DV format.

When you do the uprez in post, you are blowing up 360 lines of color subsampled DCT compressed image up to 480. Not pretty. SQUEEZE gives you a considerably better image than an post process can, and that holds true for NTSC OR PAL.

So, I have to ask. Have you ever tested SQUEEZE, and have you ever tested Instant HD? Because SQUEEZE isn't all that bad, and Instant HD isn't all that good, in my tests, anyway.

 

[ProfessorU]

Well Joshua pretty much saved me the trouble of assembling my counterargument.
Which is the weakest link? The sensor or the MiniDV tape? Based on what I've seen from Andromeda, I'm going to guess the tape.

I actually shot a wedding with two DVX cams, accidentally leaving one in letterbox and one in squeeze. In the final output, the squeeze had more visible details.

 

[Jack Daniel Stanley]

... So, I have to ask. Have you ever tested SQUEEZE, and have you ever tested Instant HD? Because SQUEEZE isn't all that bad, and Instant HD isn't all that good, in my tests, anyway.

No instant HD isn't all that good which is why I suggested don't bother with it if you have FCP. Some other NLE's blow up worse than FCP though and you might want to consider it then.

Now ... I had a DVX100A and base most of my stuff off of what was in Barry's book at the time and what the camera did at the time.

As for numbers I was going by Professor U's numbers and (from memory) His .9, 720X480 which is what NTSC is so it sounded about right to me. I know the DVX actually resolves to about 500 lines, by the tested resolution in Barry's book. What he saw when he actually pointed it at a resolution chart. (but again, I am basking it on memory).

The crux of my argument is EXACTLY the same however. Lets use some general descriptors to not get lost in the numbers.

SQUEEZE on the DVX is a fudged. Not full resolution.It is the MIDDLE of the image blown up / stretched / distorted in camera to take take up all the vertical space in the frame. It does not produce full resolution 16X9 video. Regardless if the middle area is 270, 360, or 400. Its stretched like taffy to get to the top and bottom of your 16X9 image.

But your'e saying somehow Squeeze blows up 400 lines to 480? And somehow blowing up that same center real estate in post equates to blowing up 360 lines to 500? Your saying that this is because the image gets blown up in camera prior to compression? Well if you are correct then that's a different story.

But I'll get Barry in here and he will tell us all for sure. As far I knew this was an argument that was laid to rest with the dinosaurs around here: Squeeze and Letterbox have about the same resolution with Letterbox giving you a cleaner starting point in post which may lead to a higher resolution blow up to 16X9.

We'll see if he can verify that and the following:
maybe some of my numbers are off but the following are facts as far as I know:

1) The DVX does not shoot full resolution 16X9. It cannot. Squeeze is not full resolution 16X9.

2) Your NLE or blow up software in post will make anamorphic 16X9 video EXACTLY the same way the camera does. By distorting the image using the same amount of information.
​

Now even if blowing up in post produced indentical results only, I'd still recommend doing 4:3 then letterbox / blow up in post because it will give you a cleaner starting point and more choices in terms of framing. Just do all your effects, filters, transitions, THEN blow up to 16X9. BUT the idea is that it will PROBABLY give you a better image to blow up in post because you have more processing power and a slew of algorithms to choose from with the new software out there ... at least that's what I've thought up until you've just now made me doubt it.

Barry?
....

Prof, the way the Andromeda works is because it grabs the data stream before it gets to the chips so no compression is applied. If you punched after the chips it wouldn't work. The tape is just the messenger / carrier of the data that's extremely compressed by the camera, not the problen in and of itself. The Andromeda bypasses compression and then the data stream is to big for miniDV and can't be recorded to tape.

 

[dory_breaux]

What I would do is shoot 4:3 and frame for 16:9. Then uprez with BCC.

 

[Jack Daniel Stanley]

According to these guys you are working with only 360 lines to blow up from as opposed to 400 lines blown up in camera before the sensors compress it. If this is true then squeeze t would be cool.

Maybe that's right but Barry's resolution tests resolved much lower due to the stretching distortion? Then again I don't see how the camera can resize something before it gets to the sensors, but I am by no means an engineer, or mathematician as I have already proven,

 

[Jack Daniel Stanley]

Just reading this thread
http://www.dvxuser.com/V6/showthread.php?t=92724&highlight=squeeze&page=8

looking for one of the older threads before the sever upgrade that lists the tested resolutions from Barry's book.

to clarify, never meant to say that you can't get great results with squeeze ... just saying that the popular opinion, which I have long shared is that blowing up in post or in camera works with the same size horizontal strip of information and stretches it vertically, and if you are looking for the absolute best possible resolution / image outside using an anamorphic adapter, then uprezzing in post is the way to go ... if this is not true and Josh P is right then that's a whole new kettle of fish.


PM'ed Barry .... waiting ... waiting ...

 

[Jack Daniel Stanley]

mod posts from this thread:
http://www.dvxuser.com/V6/showthread.php?t=45884

It's the same either way.

Squeeze mode take the image which would be between the black bars in letterbox mode and stretches it to fill the 720x480 area.

Shooting 4:3 and cropping in post does the same thing, in terms of resolution.

If you're making a widescreen DVD, it works out to the same.

If you're making a 4:3 DVD with letterbox, it works out to the same thing.

Shooting 4:3 gives you the option of reframing in post. Squeeze doesn't.

from the same thread

For clarification for future readers, let's spell out that "squeeze" on a DVX is very different from "squeeze" on an HVX. On a DVX, squeeze is a digital stretch process.

On an HVX, "squeeze" is true native widescreen using the full 16:9 chip.

The name is the same, the process is very different.

 

[Jack Daniel Stanley]

random, but veteran member from this thread
http://www.dvxuser.com/V6/showthread.php?t=48738

I did a quick test. I shot the same scene in 24p twice. Once in squeeze, and once in normal 4x3. With the squeeze footage i used thin line detail. The two scenes looked very close, with the squeeze footage looking slightly soft. But very close.

I just got a hold of photozoom pro. So now I can try Disjecta's method of uprezzing the letterboxed footage. In my limted testing of the software, it seemed to work pretty well, but I can't imagine doing a long project with it. Hopefully in the next few days I can figure this out once and for all.

Ok done for now ... Barry? Barry? waiting ... waiting ...

 

[Barry_Green]

Wow, squeeze vs. letterbox -- hard to believe we're still arguing this!

Okay, here's my perspective on it -- I largely agree with Jack on most of what he's written, but there is also some truth in what Joshua wrote too. I have become a much bigger fan of Squeeze than I was at the time I wrote the book.

Here's the fundamental crux: squeeze and letterbox both use the same area of the sensor, the difference is whether the DVX stretches it out or not. It's the same patch (or almost the exact same patch, there may be a couple lines' difference). You're dealing with the same image. The question is whether you stretch it out in-camera or in post.

There are arguments that can be had both ways. If you stretch it in post you have the opportunity to use dedicated software on a high-powered computer to do the extraction/interpolation, whereas if you do it in-camera you're limited to whatever the camera's own processor can manage to accomplish in the limited window of time (1/30th of a second) that it's afforded.

BUT: you're dealing with a much higher quality source in-camera (an uncompressed 4:4:4 image) for scaling, versus scaling in post where you're working with a 5:1 compressed 4:1:1 image. Even so, I doubt you lose much by doing it in post, because the image you're working with still has chroma sampling on every single line. In PAL it may be worse because you're dealing with 4:2:0 chroma, so the end results of scaling may not be as good.

Either way you're stretching the footage out by 33% vertically.

In my testing I found that you could get *slightly* (as in, barely) better results from doing the 16:9 conversion in post, but the hassle factor was so outweighed by the convenience of just shooting Squeeze in the first place, that I defaulted to using Squeeze. Heck, if you really need that last 5% of quality, you should probably be shooting on an HVX in the first place!

Okay, now, back to letterbox vs. 4:3, here's where I disagree with Jack. I would much rather shoot letterbox, which gives you a cleaner image (about maybe 5% to 10% cleaner) because it's a lot less compressed. The black bars do really contribute to a cleaner image. Not a lot, a little, but hey, I'll take it. And I very much like the idea of composing the shot right and getting it right. But, I'm in a pretty small minority there, it seems like everyone else on earth wants to tape off their screen and then letterbox in post. I'd rather just get it right the first time, and get cleaner footage to boot, but I know when I'm outvoted and this is a time.

So, which should you use? It makes very little difference. If you want the absolute cleanest best possible quality, I'd suggest shooting letterbox in camera, and doing the stretch in post in an excellent stretching program, can probably yield that. But you're kind of splitting hairs there, because shooting on an HVX in native 16:9 will be drastically superior, so... don't sweat the small stuff. If you want 16:9 footage, I say put it in Squeeze and go. You'll get probably 95% of the final quality of the stretch-in-post thing, but with absolutely no hassles.

If you want that extra 5% of quality, shoot letterbox and stretch in post.

If you want the ability to reframe vertically, tape off your screen and shoot normal 4:3 and then stretch in post.

They're all fine choices, and they all can be used for bigscreen results (as 9 Songs, The Aristocrats, Murderball, Spellbound, November, Jackass, and other films have all proved).

 

[Jack Daniel Stanley]

So I am going to try to put a bullet point list of what we know and don't know together from the various posts here:

Wow, squeeze vs. letterbox -- hard to believe we're still arguing this!

Nor could I, as I said I thought it was very old news.

Okay, here's my perspective on it -- I largely agree with Jack on most of what he's written, but there is also some truth in what Joshua wrote too. I have become a much bigger fan of Squeeze than I was at the time I wrote the book.

But I guess this is why its resurfacing, there is a shifting opinion about it.

here's my readers digest version, bullet point list, to make it easy to consider each point, and to distinguish from what is empirical and what's a good guess.

• FACT: Squeeze in camera and stretch in post use the same # of lines in the middle area. They travel an almost identical patch in the camera.

• HYPOTHESIS: The camera has less processing power and my not be able to do as good a job stretching as your computer.

• HYPOTHESIS: In camera is working with / stretching a higher quality source (uncompressed 4:4:4), vs. stretching in post which is working with a 5:1 compressed 4:1:1 image (fact). But any gain in quality would be negligible because there is still chroma sampling on every line (the hypothesis). For NTSC this probably holds true, but not so much for PAL due to the type of chroma sampling, scaling in post may be worse than scaling NTSC in post.

• FACT: Tests conducted by Barry Green using resolution charts produced SLIGHTLY (as in barely) higher resolution final video.

• FACT: UPREZZING in post takes more time than using Squeeze in camera (obviously)

Which to use summary:

1. ... don't sweat the small stuff. If you want 16:9 footage, I say put it in Squeeze and go. You'll get probably 95% of the final quality of the stretch-in-post thing, but with absolutely no hassles.

2. If you want that extra 5% of quality, shoot letterbox and stretch in post.

   and / or
   

   If you want the ability to reframe vertically, tape off your screen and shoot normal 4:3 and then stretch in post.

3. They're all fine choices, and they all can be used for bigscreen results (as 9 Songs, The Aristocrats, Murderball, Spellbound, November, Jackass, and other films have all proved).

ULTIMATELY:

They're all fine choices, and they all can be used for bigscreen results (as 9 Songs, The Aristocrats, Murderball, Spellbound, November, Jackass, and other films have all proved).

.....


INTERESTING additional letterbox info:

Okay, now, back to letterbox vs. 4:3, here's where I disagree with Jack. I would much rather shoot letterbox, which gives you a cleaner image (about maybe 5% to 10% cleaner) because it's a lot less compressed.

The black bars do really contribute to a cleaner image. Not a lot, a little, but hey, I'll take it. ...

FACT: Letterboxing in Camera will give you a 5 - 10% cleaner image vs. letteboxing in post because (I'm guessing) like cropping your footage for the web, you are sending less information over the same processing power.

 

[ullanta]

I don't know exactly whatr "5-10% cleaner is"... and I'd really like to see this empirically to be sure the DVX does indeed take good advantage of the black bar compression (I have a DVX100P, so no squeeze mode to try). My intuition is that the compression benefits of the squeezed image would trump the compression benefits of the letterbox image. But I'm just theorizing. Barry, you sound like you've actually observed this? Any chance of an example or two? I don't doubt it if it's been observed, I'm just leery of the argument that camera processing is "too stupid to stretch optimally" but "smart enough to compress optimally".

I also want to toss one more detail into the discussion here - and that's the progressive vs. interlaced issue. As a (perhaps obvious) rule I'd say to use squeeze if shooting interlaced 16:9, since the squeeze mode does not require you to vertically stretch in post, which will unduly soften interlaced footage. (This assumes the camera does "the right thing" when interlacing squeeze-mode footage). Similarly, 24p footage MIGHT have similar stretching issues on the "mixed" frames. 24pA footage, with pulldown removed, is the best candidate for stretching in post.

 

[cardmaverick]

Prof, the way the Andromeda works is because it grabs the data stream before it gets to the chips so no compression is applied. If you punched after the chips it wouldn't work. The tape is just the messenger / carrier of the data that's extremely compressed by the camera, not the problen in and of itself. The Andromeda bypasses compression and then the data stream is to big for miniDV and can't be recorded to tape.

...... Actually, the Andromeda is pulling data straight off of the cameras ADC's and then out to a computer.

I can't really say much about the cameras squeeze or letterbox modes... to spoiled with 4:4:4 HD :laugh:

 

[Jack Daniel Stanley]

...... Actually, the Andromeda is pulling data straight off of the cameras ADC's and then out to a computer.

I can't really say much about the cameras squeeze or letterbox modes... to spoiled with 4:4:4 HD :laugh:

I'm sure you are right. I obv. have no idea how its really put together. I just now it pulls it out uncompressed - which is before compression right?

 

[cardmaverick]

Yep!

Though, knowing the fact that the DVX is really an HD camera makes this discussion kinda interesting... Just goes to show its all about the signal processing.

 

[cardmaverick]

You know what... I'm curious, I think I'm gonna mess with those functions and compare them to the Andromeda output.

 

[Jack Daniel Stanley]

Haha.

If you have an Andromeda Go shoot something cool. Don't waist your time with that test, it'll be like comparing an acorn to a coconut.

 

[Barry_Green]

I don't know exactly whatr "5-10% cleaner is"... and I'd really like to see this empirically to be sure the DVX does indeed take good advantage of the black bar compression

It does, I ran the test and posted the pictures years ago.

Don't you have an HVX? You can do the same test, just compare letterbox against 4:3 and examine the central area. DV is smart enough to reallocate bits from understressed blocks to those blocks who need the bits.

Similarly, 24p footage MIGHT have similar stretching issues on the "mixed" frames. 24pA footage, with pulldown removed, is the best candidate for stretching in post.

24pA with pulldown removed is also the best candidate for squeeze mode; squeeze looks great on progressive footage but not so great on interlaced. The DVX isn't like the HVX, it doesn't scan at 60p and then interlace afterwards, it actually scans the CCD in interlaced fashion, so the stretching process has to work on interlaced footage and you end up with quite a bit lower res, something on the order of 270 lines.

DV compression is also smart enough to know how to compress progressive footage differently than interlaced footage; it either compresses the full frame or it does field by field, and full-frame is more efficient than field by field.