7D and spatial aliasing

[hjulenissen]

So I read that some thinks (or measures) that DSLR-video cameras like the 7D has:
1) More aliasing than true video cameras
2) That if the camera had had better pre-sensor filtering, then it would not have had much more resolution than SD anyways.

I find this puzzling. Canon has a multi-mega-pixel sensor, and they are probably using some sort of optical lowpassfilter to sort of adhere to Nyquists sampling theoreme (I never get used to the way in which image/video equipment always adhere to Nyquist in a "sort of" manner).

So is the problem really that Canon did not take the time/cost to do a proper downscaling in its dsp from sensor resolution to 1920x1080/1280x720? Cant they afford to run the HQ demosaicing at the required framerate? Or is it that their optical lowpassfilter is good enough for stills, but that aliasing artifacts are simply more visible in moving images?


I am also intrigued that "videophiles" are annoyed by spatial aliasing, while they seem to think that heavy temporal aliasing is no problem. Is the core of the issue that "everything that old analog equipment did is good", including jerky, aliased 24 fps movement?

 

[Ian-T]

Nahh...I think we all just can't adjust to change. We'll get over it....someday.

 

[hjulenissen]

Also, how does the 720p and 1080p modes differ when it comes to aliasing?

I think that 720p60 suits my needs better than 1080p30, if both are done properly...

-k

 

[Martti Ekstrand]

You have a aliasing test test here done by Barry Green.

http://www.dvxuser.com/V6/showthread.php?t=187165

 

[TheSoulhunter]

Well, regarding the DSLR aliasing in general, there are just two possibilitys imo:

1. Input is too sharp and the lowpassfilter isnt strong enough
or
2. The internal downscaling sucks

My bet is is that its the downscaling... Downscaling algorithms that neither blur too much, nor leave aliasing, are much more demanding (in regards to processing complexity -> CPU power / RAM / etc.) than the simple ones (which are either blurry or introduce aliasing or ringing). And as the resampling has to happen very fast (up to 60fps for 720p), they probably picked one of the simple algos!

 

[Michael Olsen]

So I read that some thinks (or measures) that DSLR-video cameras like the 7D has:
1) More aliasing than true video cameras
2) That if the camera had had better pre-sensor filtering, then it would not have had much more resolution than SD anyways.

DSLRs definitely have less empirical resolution (as a result of aliasing in the case of the 7D and the 5D) than traditional HD video cameras such as the EX-1.

With better filtering and other hardware optimized for video production, it is likely that these cameras would exceed the resolution of SD in a statistically significant way. In fact, the 7D and 5D already do so in 1080p mode. (Unfortunately, that's really nothing to brag about...)

I find this puzzling. Canon has a multi-mega-pixel sensor, and they are probably using some sort of optical lowpassfilter to sort of adhere to Nyquists sampling theoreme (I never get used to the way in which image/video equipment always adhere to Nyquist in a "sort of" manner).

Canon does have a multi-mega-pixel sensor and they are undoubtedly using an OLPF. However, the entire camera, including the OLPF, is optimized for the production of high resolution stills footage, captured at 13MP. The addition of 1080p video (which would really only require a 3MP sensor, even considering debayering) continues to be an afterthought in terms of hardware optimization.

So is the problem really that Canon did not take the time/cost to do a proper downscaling in its dsp from sensor resolution to 1920x1080/1280x720? Cant they afford to run the HQ demosaicing at the required framerate? Or is it that their optical lowpassfilter is good enough for stills, but that aliasing artifacts are simply more visible in moving images?

As was stated in a number of times in the test thread mentioned above, it seems like the performance we are seeing can come from three places.

1. The OLPF is optimized for taking high megapixel stills. When the camera is switched into video mode, the pixel binning aspect of the capture process doesn't seem to play nicely with the OLPF.

2. The actual process of pixel binning and downscaling on the camera introduces additional aliasing on its own.

3. The codec also offers room for improvement, but would require additional processing capability.

I am also intrigued that "videophiles" are annoyed by spatial aliasing, while they seem to think that heavy temporal aliasing is no problem. Is the core of the issue that "everything that old analog equipment did is good", including jerky, aliased 24 fps movement?

Well, the goal for many video users, especially digital cinematographers, is to recreate the look and feel of "old analog equipment" - 35mm film cameras. These analog cameras don't experience much in the way of spatial aliasing, you are right, but they are victims of temporal aliasing simply due to their frame rate.

Hence, I don't think anyone using any camera wants aliased or jerky footage because film wasn't aliased or jerky*. They do want 24fps with a 180º shutter because that produces the familiar appearance of film**.

There are, however, a number of individuals who are pushing ahead with the possibilities of video and believe that people will come to accept and enjoy 60fps with a 180º shutter, along with a complete lack of noise and/or grain more than film.

*Of course, there are times where people may want jerky movement or aliasing to create some sort of effect in camera.

**And of course, film was not always run at 24fps with a 180º shutter.

 

[sblfilms]

The sensor in the 7D is an 8FPS burst sensor at full resolution. That means its real sustained FPS (if you have media fast enough to right to it) is around 6. So the sensor is 75% slower than is needs to be to record 24fps material full res. Canon's solution to this is to simply read out every third horizontal line and then scale the remainder to 1080p. In 720p mode the appear to read every fourth line (which explains the increasing aliasing going for 1080p to 720p).

The only real solution to this is for Canon to engineer faster sensors. The problem is that they can't stick a full res DSLR sensor that runs 24fps into a DSLR body. There just isn't adequate room for cooling.

The truth that many don't want to face is that Red is doing Super35MM high res sensor cameras for about as cheap as you can go and Canon, Panasonic, JVC, etc are not going to put out their version of the Red for any less than what Red is selling their cameras for.

So just enjoy the 7D and future DSLRs for what they are. I've really fallen in love with the little bugger and though I fully plan to get a tricked out Scarlett S35 next year the 7D and future models will probably continue to be a staple of my camera bag for years to come.

 

[yoclay]

This last post sounds about right. Reality being what it is.

 

[Barry_Green]

The only real solution to this is for Canon to engineer faster sensors.

This is the core of the whole issue, yes. It's an 8fps sensor, and so compromises have to be made to get it to run at 24 or 60fps. And those compromises bring the aliasing.

On the other hand, it costs $1700. So pick your compromise: save money and get aliasing, or spend money and get a Red One (or someday a Scarlet) and avoid aliasing.

The problem is that they can't stick a full res DSLR sensor that runs 24fps into a DSLR body. There just isn't adequate room for cooling.

The 8fps sensor they have already has overheating issues; tripling or 8x the speed would introduce presumably major heat issues, yes.

The truth that many don't want to face is that Red is doing Super35MM high res sensor cameras for about as cheap as you can go

Yep.

So just enjoy the 7D and future DSLRs for what they are.

Yep.

 

[hjulenissen]

The sensor in the 7D is an 8FPS burst sensor at full resolution. That means its real sustained FPS (if you have media fast enough to right to it) is around 6. So the sensor is 75% slower than is needs to be to record 24fps material full res. Canon's solution to this is to simply read out every third horizontal line and then scale the remainder to 1080p. In 720p mode the appear to read every fourth line (which explains the increasing aliasing going for 1080p to 720p).

The only real solution to this is for Canon to engineer faster sensors. The problem is that they can't stick a full res DSLR sensor that runs 24fps into a DSLR body. There just isn't adequate room for cooling.

The truth that many don't want to face is that Red is doing Super35MM high res sensor cameras for about as cheap as you can go and Canon, Panasonic, JVC, etc are not going to put out their version of the Red for any less than what Red is selling their cameras for.

So just enjoy the 7D and future DSLRs for what they are. I've really fallen in love with the little bugger and though I fully plan to get a tricked out Scarlett S35 next year the 7D and future models will probably continue to be a staple of my camera bag for years to come.

Thank you,this make sense to me.

Capturing every 1 out of 3 or 1 out of 4 lines would effectively:
1)Mean that vertically, effectively no relevant OLPF is applied and the 33%/25% "coverage" is more like a point-sampler than a Nyquist sampler.
2)Horizontally, full resolution could be digitized and filtered more or less ideally

Given that the bayer-patter is some permutation of:
red-green
green-blue

how is it possible to get a full color image if reading every 4th line? Would you not get only green/red samples or only blue/green samples?

I see an application for "interlacing" here: if alternating the lines that are read, some spatial aliasing could be attenuated, relying on some assumptions about normal scenes and clever deinterlacing?

Canon have a "soft" 100mm lense. How would that perform as bandlimiting filter?

-k

 

[hjulenissen]

Downscaling algorithms that neither blur too much nor leave aliasing are much more demanding (in regards to processing complexity -> CPU power / RAM / etc.) than the simple ones (which are either blurry or introduce aliasing or ringing). And as the resampling has to happen very fast (up to 60fps for 720p) they probably picked one of the simple algos!

I think that your explanation is unlikely.

High-quality linear scaling of rgb or YCbCr is actually not that expensive. It can be done in realtime in home-theater PCs for HD content.It can be done in ok-ish flatscreen tvs and projectors. For embedded dsp or ASICs, the same algorithms can be applied highly optimized (that is basically what dsps are designed to do, unlike x86 cpus).

I think that explanations about sensor bandwidth/heat, and the "simply not optimized for this task" is more likely. I dont know the Canon processing hardware, but this could be a strong motivation for going one step in the direction of general purpose dsp instead of fixed function ASICs, meaning that the same "horsepower" could load new code for doing nice things for video instead og stills. Of course,that would offset other trade-offs in speed and quality and battery-life.

-h

 

[hjulenissen]

The only real solution to this is for Canon to engineer faster sensors. The problem is that they can't stick a full res DSLR sensor that runs 24fps into a DSLR body. There just isn't adequate room for cooling.

This is a pet dream of mine:
If the architecture is flexible enough, being able to swap sensor for the user would be great. It would also be a nice source of income for Canon & Co.

For a 7D-ish SLR, I would really like having:
1) 18megapixel monochrome still-image sensor
2) 10 megapixel color still-image sensor
3) 3 megapixel color video sensor

I guess that making a mechanical arrangement that was sturdy and precise enough would be a major obstacle. But from my understanding of dsp, it should be possible to re-use most resources for all those sensors.

A 3 megapixel sensor +/- should be able to do good 1080p at high framerates without being unreasonable in terms of bandwidth.


On second though, this problem wouldbe solved if it was possible to insert an optimized OLPF. Sprsely point-sampling is no problem if you apply the right pre-filter.

edit:
having read the excellent article. I just find it very hard to grasp that the true capture is closer to SD than HD. I see that a HD imager with insufficient OLPF would have severe aliasing, but fail to see how these imagers cannot pass the frequency range between SD and HD.

BTW, what do you mean by a 180-degree shutter?

-h

 

[TheSoulhunter]

I think that your explanation is unlikely.

High-quality linear scaling of rgb or YCbCr is actually not that expensive. It can be done in realtime in home-theater PCs for HD content.It can be done in ok-ish flatscreen tvs and projectors. For embedded dsp or ASICs, the same algorithms can be applied highly optimized (that is basically what dsps are designed to do, unlike x86 cpus).

Well, even considering it's build and optimized for just this task (in difference to a general purpose architecture like x86) it's still processing 17.9M pix (7d) -VS- processing 2.1M pix (the HDTV in your example, which uses special DSPs as well, not x86 or similar, no?). Big difference... But yes, this "interlaced" readout theory makes sense as well. Do we have some official info about it, or is it also just "guessed"?

 

[hjulenissen]

Well, even considering it's build and optimized for just this task (in difference to a general purpose architecture like x86) it's still processing 17.9M pix (7d) -VS- processing 2.1M pix (the HDTV in your example, which uses special DSPs as well, not x86 or similar, no?). Big difference...

The tv set would have to process 60 frames a second, while the SLR only does 30fps. Further, the cost of scaling is a product of input pixelrate as well as output pixel rate. But I see your point, my example was less than convincing.

A true 1080p video camera would have far from ideal spatial lowpass filtering. It is probably more similar to a simple box-car averaging (+OLPF). If the quality of a high-end 1080p camera is the goal, scaling a pre-processed 18 mpix rgb image to 1080p could probably be done with something as simple as neighborhood averaging. The JPEG encoder in 7D does 8x8 pixel averaging at 18megapixel, 6fps as a tiny, tiny fraction of its operations...

-h