Kodak's Truesense Imaging creates 4K 4/3 chip

[Samuel H]

Seems they presented three sensors. I referred to the KAI-04070 - with a strange square design 2048 x 2048. But I checked the size and still not real Micro4/3: Only 15,2 mm wide. That's BlackMagic area, right?

Yes, they presented several models, but they are all relatively small or have very limited DR.
The KAI-04070, once you crop it to video aspect ratio, has a crop factor of around 2.4x, so, slightly smaller than the original BMC (15.2mm wide vs 15.8mm on the BMC)

 

[Samuel H]

This is not true. The luminance resolution of a Bayer image is the full pixel resolution of the whole sensor as a monochrome sample. Chroma resolution is lower than 3-chip or Foveon sensors. But sophisticated debayer processing is a 4 vector matrix: Luminance + RGB each as discrete value sets. The Bayer pattern process is the closest electronic mimic of the way the human eye resolves both luminance and color.
The problem with its use in video cameras is the crude debayer processes needed to meet real time frame rate requirements with limited processing power. Hence the resulting artifacts and excessive loss of luminance and chroma resolution.

Regardless of whether a camera uses one chip or 3, oversamples or not, the optical luminance resolution of the final image is Nyquist limited by the output format to about 85% of the format pixel resolution unless one is willing to suffer significant aliasing artifacts. For 1080p delivery formats, including HD and 2k 1:85 the practical limit to vertical resolution is around 850 TV lines or 425 clean line pairs per height. Very few cameras actually make this, Bayer or not.

Correct.

Most of the softness of debayered-at-full-sensor-resolution material comes not from the debayering methods (which are pretty much awesome nowadays) but from the antialiasing filter (OLPF).

In fact, if you want proof, you can look at the 5D3-RAW. That's storing a 1920x1080 RAW image, and after debayering it you get really really sharp 1920x1080 video. Because the original RAW is not compromised by a super-strong OLPF.

 

[Razz16mm]

Correct.

Most of the softness of debayered-at-full-sensor-resolution material comes not from the debayering methods (which are pretty much awesome nowadays) but from the antialiasing filter (OLPF).

In fact, if you want proof, you can look at the 5D3-RAW. That's storing a 1920x1080 RAW image, and after debayering it you get really really sharp 1920x1080 video. Because the original RAW is not compromised by a super-strong OLPF.

The OLPF is an optical filter in front of the sensor. It affects everything the camera shoots equally regardless of output format. The camera either has one or it doesn't. The difference between 5D raw video and the camera's H.264 video output is strictly the difference in Debayer processing quality and video encoding quality between what the camera can process internally compared to a much more powerful editing workstation with superior software.

 

[Razz16mm]

Yes, they presented several models, but they are all relatively small or have very limited DR.
The KAI-04070, once you crop it to video aspect ratio, has a crop factor of around 2.4x, so, slightly smaller than the original BMC (15.2mm wide vs 15.8mm on the BMC)

If you think the potential DR of these is inferior tell me how Sony is getting 14 stops DR from a sensor with 57dB spec noise ratio in the F5 and F55.

 

[Samuel H]

No idea where you got that spec from, but 57dB is 9.5 stops. If it is right, then the sensor must have some HDR trick built in, like dual-gain (like Alexa and BMC) or something similar.
edit: got it, I see the 57dB figure in the F55 manual. It refers to S/N ratio, not to DR. Different things, both measured in the same units. For example, this Chinese sensor was presented today: http://image-sensors-world.blogspot.com.es/2013/09/superpix-publishes-flyer-of-its-dslr.html


Re: OLPF: yes, I know debayered won't ever look as good as shooting RAW and doing the debayering in the PC. My point was that if a RAW 2K image is softer than 2K video debayered from a 3K sensor, it is usually because of the OLPF, not because the debayering creates softness. Downsample first, debayer later, and the image can be very very sharp.

 

[Razz16mm]

No idea where you got that spec from, but 57dB is 9.5 stops. If it is right, then the sensor must have some HDR trick built in, like dual-gain (like Alexa and BMC) or something similar.
edit: got it, I see the 57dB figure in the F55 manual. It refers to S/N ratio, not to DR. Different things, both measured in the same units. For example, this Chinese sensor was presented today: http://image-sensors-world.blogspot.com.es/2013/09/superpix-publishes-flyer-of-its-dslr.html


Re: OLPF: yes, I know debayered won't ever look as good as shooting RAW and doing the debayering in the PC. My point was that if a RAW 2K image is softer than 2K video debayered from a 3K sensor, it is usually because of the OLPF, not because the debayering creates softness. Downsample first, debayer later, and the image can be very very sharp.

The 57dB figure is straight off of Sony's spec sheet for the F5/F55 with noise reduction off. You can easily pick up 1-2 stops of DR with good noise reduction processing, but 4 or more seems to be stretching it a bit.

 

[harddrive]

Re: OLPF: yes, I know debayered won't ever look as good as shooting RAW and doing the debayering in the PC. My point was that if a RAW 2K image is softer than 2K video debayered from a 3K sensor, it is usually because of the OLPF, not because the debayering creates softness. Downsample first, debayer later, and the image can be very very sharp.

Let's deal with basic deBayering and the OLPF separately.

The purpose of the OLPF is to lower aliasing levels. Since real resolution turns into aliases abruptly (at the Nyquist point) the ideal characteristics of an OLPF would be to pass all detail below that spatial resolution and none above it. Unfortunately, such is not possible. It's a gradual roll off, though much sharper than such as a diffusion filter in front of the lens. Consequently, there has to be a compromise - either design the OLPF to inhibit most aliasing and accept softening of wanted detail OR accept aliasing and get more real detail.

But to ignore the OLPF for a moment, then however it's done, the purpose of deBayering is to reconstruct a normal colour image from a matrix of red, green and blue samples. For Bayer, the simplest way is to take the samples in blocks of four and treat each block as a pixel - there will be discrete R,G,B values directly from photosite values. Advantage is very simple processing - downside is limiting resolution to half the sensor native resolution. (This is how the C300 works.)

DeBayering is normally taken to mean a more sophisticated process whereby for (say) a red photosite, the system interpolates what the green and blue values should have been from the surrounding green and blue photosites and hence outputs an R,G,B pixel. "Interpolates" is a mathematical term for "guesses" or "averages" - though yes, some interpolation algorithms are far more sophisticated than others. If you really want examples then try http://www.stark-labs.com/craig/resources/Articles-&-Reviews/Debayering_API.pdf . The point is that although some algorithms are far better than others, none is perfect. Whilst even the poorest may give you better than the half resolution of the 2x2 method above, none can give you luminance resolution equivalent to if you just scrapped the Bayer filtration and used the same sensor in a monochrome mode. The normally accepted figures for "how good then?" obviously depend on many things, including the algorithm used, but "about 80%" is the normally accepted ball park figure.

Note also the resolution doesn't stop abruptly - it tails off, and this brings up the whole subject of modulation transfer function. Hence, at what figure for mtf do you consider resolution to have faded away?

These represent the MAXIMUM figures that can be obtained from a Bayer array. Once you put any OLPF in front, it can only soften, lower the mtf at any given figure. How much depends on the compromise that's prepared to be accepted between aliasing and sharpness. And trying to form an opinion simply by eye judgement alone is dangerous. The whole point of aliasing is that it can cause far more problems with video than still images, and consequently needs to be suppressed more. (It's not the look of the image, it's the effect moving aliases can have on compression engines further down the chain.)

 

[Razz16mm]

Let's deal with basic deBayering and the OLPF separately.

The purpose of the OLPF is to lower aliasing levels. Since real resolution turns into aliases abruptly (at the Nyquist point) the ideal characteristics of an OLPF would be to pass all detail below that spatial resolution and none above it. Unfortunately, such is not possible. It's a gradual roll off, though much sharper than such as a diffusion filter in front of the lens. Consequently, there has to be a compromise - either design the OLPF to inhibit most aliasing and accept softening of wanted detail OR accept aliasing and get more real detail.

But to ignore the OLPF for a moment, then however it's done, the purpose of deBayering is to reconstruct a normal colour image from a matrix of red, green and blue samples. For Bayer, the simplest way is to take the samples in blocks of four and treat each block as a pixel - there will be discrete R,G,B values directly from photosite values. Advantage is very simple processing - downside is limiting resolution to half the sensor native resolution. (This is how the C300 works.)

DeBayering is normally taken to mean a more sophisticated process whereby for (say) a red photosite, the system interpolates what the green and blue values should have been from the surrounding green and blue photosites and hence outputs an R,G,B pixel. "Interpolates" is a mathematical term for "guesses" or "averages" - though yes, some interpolation algorithms are far more sophisticated than others. If you really want examples then try http://www.stark-labs.com/craig/resources/Articles-&-Reviews/Debayering_API.pdf . The point is that although some algorithms are far better than others, none is perfect. Whilst even the poorest may give you better than the half resolution of the 2x2 method above, none can give you luminance resolution equivalent to if you just scrapped the Bayer filtration and used the same sensor in a monochrome mode. The normally accepted figures for "how good then?" obviously depend on many things, including the algorithm used, but "about 80%" is the normally accepted ball park figure.

Note also the resolution doesn't stop abruptly - it tails off, and this brings up the whole subject of modulation transfer function. Hence, at what figure for mtf do you consider resolution to have faded away?

These represent the MAXIMUM figures that can be obtained from a Bayer array. Once you put any OLPF in front, it can only soften, lower the mtf at any given figure. How much depends on the compromise that's prepared to be accepted between aliasing and sharpness. And trying to form an opinion simply by eye judgement alone is dangerous. The whole point of aliasing is that it can cause far more problems with video than still images, and consequently needs to be suppressed more. (It's not the look of the image, it's the effect moving aliases can have on compression engines further down the chain.)

+1. Nice summary. I will add that the output format itself, in most cases 1080p HD, also has a practical Nyquist limit beyond which visible aliasing artifacts will occur too. It doesn't matter once you reach the format limit what the input process is.

When aliasing artifacts do occur they pollute resolved details well below the Nyquist limit. You can often get more cleanly resolved detail with an OLPF than without and a bit of judiciously applied sharpening can do wonders. This is especially true if you are working directly from Bayer raw footage and not from camera compressed video.

 

[harddrive]

When aliasing artifacts do occur they pollute resolved details well below the Nyquist limit. You can often get more cleanly resolved detail with an OLPF than without and a bit of judiciously applied sharpening can do wonders.

To really understand aliasing, you need to look at it in terms of single frequencies at any one time - whether that be for audio, something like a fine grid pattern, or time related aliasing (think wagon wheels going backwards in the cinema).

At this point, the key thing to realise (and it took me a long time) is that such are either reproduced correct (same as the original) or they are wrong (aliases). You don't get a mixture. If you put a single frequency tone into a digital sampling circuit, you'll either get the same tone out - or an alias at a different frequency. What you WON'T get is a mixture of the two.


Think of a single dimensional "sensor" with 100 discrete elements. It should be obvious that the maximum it can resolve is a pattern of 100 dots, or 50 black-white pairs if you like. That's the finest pattern it can output, period. If you put in a pattern of (say) 5 pairs, it's obvious it won't have any trouble resolving it accurately - each output "dot" will consist of 10 white, 10 black "pixels". Make the pattern finer, and it should likewise cope. But what about if we put in a pattern finer than 100 dots? Say 120, or 60 pairs?

One thing it CANNOT do is output any pattern finer than 100 dots. In practice, I'd expect to see a rather murky pattern of 80 dots or 40 pairs - aliasing predicts it will reflect around the Nyquist point. What you won't get is a pattern of 120 with an alias as well.

In audio, if you sample at 10,000 times per second, the highest frequency on the output will be 5kHz, end of story, no matter what you put in. Put a frequency up to 5kHz in, and it should be fairly accurately relayed. Go above that, and it will alias down - so 5.1kHz becomes an output 4.9kHz, 5.2kHz becomes 4.8 etc.

For images, the "input" is typically more like an edge, which translates to a range of frequencies mathematically. If the resolution cuts off smoothly, you should see the edge slightly softened, if there is aliasing present it's more likely to have a bit of a ring to it.

Sorry to get off topic, but the basic theory is widely misunderstood.

 

[Razz16mm]

To really understand aliasing, you need to look at it in terms of single frequencies at any one time - whether that be for audio, something like a fine grid pattern, or time related aliasing (think wagon wheels going backwards in the cinema).

At this point, the key thing to realise (and it took me a long time) is that such are either reproduced correct (same as the original) or they are wrong (aliases). You don't get a mixture. If you put a single frequency tone into a digital sampling circuit, you'll either get the same tone out - or an alias at a different frequency. What you WON'T get is a mixture of the two.


Think of a single dimensional "sensor" with 100 discrete elements. It should be obvious that the maximum it can resolve is a pattern of 100 dots, or 50 black-white pairs if you like. That's the finest pattern it can output, period. If you put in a pattern of (say) 5 pairs, it's obvious it won't have any trouble resolving it accurately - each output "dot" will consist of 10 white, 10 black "pixels". Make the pattern finer, and it should likewise cope. But what about if we put in a pattern finer than 100 dots? Say 120, or 60 pairs?

One thing it CANNOT do is output any pattern finer than 100 dots. In practice, I'd expect to see a rather murky pattern of 80 dots or 40 pairs - aliasing predicts it will reflect around the Nyquist point. What you won't get is a pattern of 120 with an alias as well.

In audio, if you sample at 10,000 times per second, the highest frequency on the output will be 5kHz, end of story, no matter what you put in. Put a frequency up to 5kHz in, and it should be fairly accurately relayed. Go above that, and it will alias down - so 5.1kHz becomes an output 4.9kHz, 5.2kHz becomes 4.8 etc.

For images, the "input" is typically more like an edge, which translates to a range of frequencies mathematically. If the resolution cuts off smoothly, you should see the edge slightly softened, if there is aliasing present it's more likely to have a bit of a ring to it.

Sorry to get off topic, but the basic theory is widely misunderstood.

The most objectionable aliasing artifacts like chroma moire are the result of interference patterns between two different physical spatial frequencies, the sensor pattern and the image detail content. Digital encoding doesn't really impact that. It is there in an analog systems too. With analog NTSC systems it was interference between scan line spatial frequency and image detail patterns. The result is like a diffraction grating that splits light into higher and lower frequency components.

 

[Samuel H]

The 57dB figure is straight off of Sony's spec sheet for the F5/F55 with noise reduction off. You can easily pick up 1-2 stops of DR with good noise reduction processing, but 4 or more seems to be stretching it a bit.

Yes, I've seen it, but it refers to a completely different thing. As I said above:
edit: got it, I see the 57dB figure in the F55 manual. It refers to S/N ratio, not to DR. Different things, both measured in the same units. For example, this Chinese sensor was presented today: http://image-sensors-world.blogspot.com.es/2013/09/superpix-publishes-flyer-of-its-dslr.html

the olpf and debayering discussion is too long, I'll read it later

 

[Razz16mm]

Yes, I've seen it, but it refers to a completely different thing. As I said above:
edit: got it, I see the 57dB figure in the F55 manual. It refers to S/N ratio, not to DR. Different things, both measured in the same units. For example, this Chinese sensor was presented today: http://image-sensors-world.blogspot.com.es/2013/09/superpix-publishes-flyer-of-its-dslr.html

the olpf and debayering discussion is too long, I'll read it later

S/N ratio an DR are essentially the same thing. Sony's spec refers to the S/N ratio with noise reduction turned off, essentially the same condition that the S/N ratio of the Kodak spec is defined under. The full well clip point doesn't change, but effective noise reduction does reduce the noise floor, 6-12dB noise reduction when gain balancing the camera is pretty easy to achieve, effectively extending DR by two stops. Getting much more than that though is real tricky. The fact that the sensor or camera maker posts a DR spec different from S/N noise ratio just means the sensor can detect light over the broader range, not that what is captured would be cleanly separable from the noise floor enough to be useful for high quality imaging absent some very sophisticated processing.

 

[combatentropy]

Someone else posted about Digital Bolex or Ikonoshop but I'm interested in an s35 like that. Is anyone else?

Yes

 

[Samuel H]

S/N ratio an DR are essentially the same thing. Sony's spec refers to the S/N ratio with noise reduction turned off, essentially the same condition that the S/N ratio of the Kodak spec is defined under. The full well clip point doesn't change, but effective noise reduction does reduce the noise floor, 6-12dB noise reduction when gain balancing the camera is pretty easy to achieve, effectively extending DR by two stops. Getting much more than that though is real tricky. The fact that the sensor or camera maker posts a DR spec different from S/N noise ratio just means the sensor can detect light over the broader range, not that what is captured would be cleanly separable from the noise floor enough to be useful for high quality imaging absent some very sophisticated processing.

A couple of quotes from here: http://www-isl.stanford.edu/~abbas/group/papers_and_pub/isscc02_tutorial.pdf
* SNR is the ratio of the input signal power to the average input referred noise power, and is typically measured in dBs
* Dynamic range (...) is defined as the ratio of the largest nonsaturating input signal to the smallest detectable input signal

To me, they are closely related (probably so much so that it is conceivable they could be considered the same in very simple sensor designs) but not the same.
In any case, the fact that 57dB is 9.5 stops, but the camera has much more DR than that, is already telling me that it's not measuring the same thing.

(the alternative explanation would be that Sony is quoting SNR as DR in terms of power instead of voltage, but that would mean 57 dB is 19 stops, and they don't get that much either; plus, no camera manufacturer quotes it that way, because it is a smaller dB number and marketing guys like large numbers)

 

[Jim Martin]

As far as I know, the Canon 4K chip in the 3 "C" series cameras is the only chip not being de-bayered. Also, the original post was asking for a 2K only camera.....and that was announced last week...the Arri Amira.

Jim Martin
Filmtools.com

 

[Razz16mm]

A couple of quotes from here: http://www-isl.stanford.edu/~abbas/group/papers_and_pub/isscc02_tutorial.pdf
* SNR is the ratio of the input signal power to the average input referred noise power, and is typically measured in dBs
* Dynamic range (...) is defined as the ratio of the largest nonsaturating input signal to the smallest detectable input signal

To me, they are closely related (probably so much so that it is conceivable they could be considered the same in very simple sensor designs) but not the same.
In any case, the fact that 57dB is 9.5 stops, but the camera has much more DR than that, is already telling me that it's not measuring the same thing.

(the alternative explanation would be that Sony is quoting SNR as DR in terms of power instead of voltage, but that would mean 57 dB is 19 stops, and they don't get that much either; plus, no camera manufacturer quotes it that way, because it is a smaller dB number and marketing guys like large numbers)

If the camera's noise floor in dB is higher than the lowest input level in dB the sensor can detect, the input can't be distinguished from the noise unless there is effective noise reduction. Black shading calibration for instance subtracts camera generated noise from actual sensor input to extend usable dynamic range. The actual dB range between white clip and noise floor defines usable DR vs theoretical DR.

 

[harddrive]

S/N ratio an DR are essentially the same thing.

No...... Dynamic range is the difference between brightest and darkest inputs to the chip with detail still resolved - signal to noise in this context is normally taken to mean the difference between max signal level and noise level ON THE OUTPUT.

The reason for the difference is due to the camera having to compress the tonal range to make it acceptable to the eye. Hence the camera may accept an INPUT dynamic range of maybe 13 stops, but after processing that gets condensed into maybe 6-7 stops worth on the OUTPUT.

At any one time the eye can only resolve something like 6 stops of dynamic range in a scene. It seems to be a lot more - but that's because the eye/brain dynamically adjusts as your attention moves around the scene, though you aren't consciously aware of it. Hence, if looking at a scene with deep shadows under trees and bright clouds, your eye won't be able to cope with detail in both at the same time - but as you look from one to the other your vision adapts. Film a scene and replay it and it's different. This is the function of such as the "knee" in the cameras response curve. A doubling of the input level will correspond to less on the output. There will be a difference - so detail is preserved in the highlights, it's not clipping - but it's condensed relative to the input.

Another way of looking at it is that the 57dB figure refers to the camera output. The s/n at the output of the chip is likely to be much better, but will be lowered inevitably via the camera processing. This is why 8 bit is fully acceptable for coded video, but you need at least 12 bit for the sensor A-D conversion. (And recording RAW video.)

 

[atarijedi]

As far as I know, the Canon 4K chip in the 3 "C" series cameras is the only chip not being de-bayered. Also, the original post was asking for a 2K only camera.....and that was announced last week...the Arri Amira.

Jim Martin
Filmtools.com

It is being debayered, just not in the traditional sense. They don't interpolate colours for each pixel based on its neighbours, they combine 4 pixels into 1.

 

[Samuel H]

No...... Dynamic range is the difference between brightest and darkest inputs to the chip with detail still resolved - signal to noise in this context is normally taken to mean the difference between max signal level and noise level ON THE OUTPUT.

The reason for the difference is due to the camera having to compress the tonal range to make it acceptable to the eye. Hence the camera may accept an INPUT dynamic range of maybe 13 stops, but after processing that gets condensed into maybe 6-7 stops worth on the OUTPUT.

That makes sense to me, thanks for the clarification.

 

[Razz16mm]

No...... Dynamic range is the difference between brightest and darkest inputs to the chip with detail still resolved - signal to noise in this context is normally taken to mean the difference between max signal level and noise level ON THE OUTPUT.

The reason for the difference is due to the camera having to compress the tonal range to make it acceptable to the eye. Hence the camera may accept an INPUT dynamic range of maybe 13 stops, but after processing that gets condensed into maybe 6-7 stops worth on the OUTPUT.

At any one time the eye can only resolve something like 6 stops of dynamic range in a scene. It seems to be a lot more - but that's because the eye/brain dynamically adjusts as your attention moves around the scene, though you aren't consciously aware of it. Hence, if looking at a scene with deep shadows under trees and bright clouds, your eye won't be able to cope with detail in both at the same time - but as you look from one to the other your vision adapts. Film a scene and replay it and it's different. This is the function of such as the "knee" in the cameras response curve. A doubling of the input level will correspond to less on the output. There will be a difference - so detail is preserved in the highlights, it's not clipping - but it's condensed relative to the input.

Another way of looking at it is that the 57dB figure refers to the camera output. The s/n at the output of the chip is likely to be much better, but will be lowered inevitably via the camera processing. This is why 8 bit is fully acceptable for coded video, but you need at least 12 bit for the sensor A-D conversion. (And recording RAW video.)

This is exactly what takes place when video is encoded in camera.