F5/F55 Sensor resolution and bayer pattern

[Steve Castle]

These are 4k raw cameras. Supersampling works for debayered outputs and implies downscaling. It is very difficult to scale RAW bayer data, binning is possible but requires a sensor with 4x the resolution. That said it is not impossible that RAW scaling could be done in some new proprietary way, since it seems the F65 does it in some way by offering 4k and 8k raw recording from its effectively 6k sensor. But the likelihood that sony is scaling 4.5k bayer data down to 4k is incredibly low. More likely such an operation would introduce more artifacts than it would increase resolution.

We see supersampling very commonly in the video market. Again, one example is the F3, we have a 3.34MP bayer that outputs 1080p. Even cheap 1080p camcorders use 3.2MP sensors these days. As we've seen with F3s put in front of resolution charts, we know that the sensor is in fact supersampling. We don't get resolution representative of a 1920x1080 minus the bayer penalty. Technically its not very remarkable given the four-figure price we are talking about.

As for raw, you don't bin raw, it should be whatever sensor data you receive off sensor. But Sony uses conservative language in how they market their F cameras. For the F65 they marketed it as 16-bit 4K raw, when in fact the raw data should be 8K2K raw. For the F5/F55 we should see the raw to essentially be 11.6MP raw unless that entire area is just a dead-area. Logically though, being that these S35 sensors are only being seen in these non-consumer cameras there is no reasonable reason to create a unused area that takes 24% the size sensor. For the F55 in particular, as its a global-shutter sensor only seen in that one camera, they redesigned the entire sensor just to waste that extra space?

Also, just FYI, you can bin in just x or y axis; you don't need 4 pixels (The Canon 5D is an example). Also, the F65 is not effectively 6K, its 8K. But using horizontal resolution to measure resolution of a sensor has to be one of the worst conventions that has happened recently. Its a 8K2K 20MP sensor. Its designed for 4K output, it can output 6K & 8K only through interpolation. Its not just a rotated bayer, that's and erroneous oversimplification.

Either way, we should see more information next week.

 

[Kolor-Pikker]

Also, just FYI, you can bin in just x or y axis; you don't need 4 pixels (The Canon 5D is an example). Also, the F65 is not effectively 6K, its 8K. But using horizontal resolution to measure resolution of a sensor has to be one of the worst conventions that has happened recently. Its a 8K2K 20MP sensor. Its designed for 4K output, it can output 6K & 8K only through interpolation. Its not just a rotated bayer, that's and erroneous oversimplification.

I do wonder then, why is it that all resolutions up to 1080p use the vertical size for measurement, but past that its 2K/4K/6K horizontally? The width is completely dependent on the format, you could have a 1.77:1 or 1.85:1 frame, but the height is always the same (on the RED you can go up to 2:1 before having to crop vertically, for example).

 

[Steve Castle]

I do wonder then, why is it that all resolutions up to 1080p use the vertical size for measurement, but past that its 2K/4K/6K horizontally? The width is completely dependent on the format, you could have a 1.77:1 or 1.85:1 frame, but the height is always the same (on the RED you can go up to 2:1 before having to crop vertically, for example).

The reasons are completely in marketing. 2160p, which 4K really is, doesn't sound very marketable. It makes more sense, as you say, as if you use vertical resolution, regardless of the resolution, it would always be 2160p. When you take 4K and reduce it down to 16:9 which is pretty much the screen size of very TV made nowadays, it becomes 3840x2160. Which is why you then have people saying 16:9 aspect ratio of 4K really isn't "4K".

The thing is, when people hear 2160p they only think resolution has only doubled from 1080p, when in fact it has quadrupled; "4K" sounds more like it has quadrupled over 1080p.

Now then, there is 8K, which is 33 Megapixels even in 16:9 aspect ratio. Its actually 16 times the resolution of 1080p and 4 times of 4K (8.3MP at 16:9). The gap between 4K and 8K is much bigger than it sounds as well. Whenever they finally start releasing 8K TVs, they'll need to think of another more marketable name as currently both 4K & 8K are bundled in 'Ultra HD".

 

[starcentral]

There's no proof yet or confirmation that the F55 is using Q67 arrangement, and obviously not something Sony wants to talk about at this time:

 

[bruceallen]

These sensors are S35 and not a 4:3.

Um, my suggestion that the other megapixels were reserved for a 4:3 anamorphic mode did have a smiley face after it.

Though if you read the FS700 brochure on Sony's site (go to Resources):
http://pro.sony.com/bbsc/ssr/product-NEXFS700UK/

They pretty clearly state that the sensor's total resolution is 4352x2662. Which is a 1.63 aspect ratio.

And also coincidentally is exactly 11.6 megapixels.

Of course, if the pixels aren't square, or are in an odd configuration, they could still have an oversampling sensor that is exactly s35 aspect ratio, so you could still be right.

A crop would be pointless in a dedicated S35 sensor, and would mean a noticeable crop factor from a standard S35. Let's put it this way, we have 2.75MP wasted, that delta is larger than the resolution of 1080P. As an engineer you would want to either fill the area and get larger photosites or, as mentioned, supersample it. If its just regular old bayer you also don't want to have exactly the target output resolution as you lose a considerable amount of data during debayering. You would want to oversample, this has been common practice for even low-priced 1080p CMOS camcorder (which usually had a ~3.2MP sensor to make a 1080p image).

I do agree that you want to oversample. But that's why Sony wants you to buy the F65 if you want picture-perfect 4K. That's why RED wants you to buy an Epic (5K) or Dragon (6K) sensor.

But hey, RED ONE didn't oversample initially - they enabled 4.5K mode later. And Canon's C500 doesn't oversample. Scarlet doesn't oversample at 24fps.

IMHO, Sony is saying "hey, if you want 4K, buy the F5 or F55. If you want perfect 4K, buy the F65."

It's not like there will be a ton of folks pixel-peeping the 4K and saying "hey, it's only 3.2K resolved!" By the time 4K gets really commonplace (at least 3 years), it'll likely be time for Sony to bring out their next camera. THEN they can give the lower end models supersampling. My 2c.

As for the FS700, we have no clue as it currently only records at 1080p only. To do 4K it seems you need a pricey recording unit, meaning a supersampled 4K would certainly be justified given the price you would need to pay.

Actually, you can get a pretty good idea of what it will look like at 4K if you take a JPEG still with it.

Check this out:
http://www.dvxuser.com/V6/showthread.php?286211-Still-Mode-on-FS700

IMHO the picture quality is good, but not so good that it looks like they supersampled by a whole lot. It looks like a normal Bayer sensor image. But it could be Q67 or ClearVid or whatever - they all look roughly similar at a pixel level.

But yeah... if RAW mode turns out to be 4352x2662 and not 4096x2160, I certainly won't be complaining!

Again, my bet is still on RAW mode being 4096x2160, no oversampling, same color filter arrangement between FS700, F5 and F55 (only change is using F65 color filter dyes for F55).

Bruce Allen
www.boacinema.com

 

[starcentral]

It's not like there will be a ton of folks pixel-peeping the 4K and saying "hey, it's only 3.2K resolved!" By the time 4K gets really commonplace (at least 3 years), it'll likely be time for Sony to bring out their next camera. THEN they can give the lower end models supersampling.

It's true what you say and afterall the original RED One wasnt even 3.2k lines resolved and look at all the movies shot with it, printed to film, and even projected.

As for pixel-peeping a lot of this speculation is just fun and really it would be a little exciting to know for sure how these new EXMOR sensors stack up to RED's existing 4k sensors since hopefully that's the cameras these are up against.

But when we honestly stop and think about what these new F's are offering, they truly are amazing; high ISO levels, global shutter (for the F55 at least), simultaneous internal and external recording, various resolutions and bitrate options, the list goes on and on... I can't wait to see these in the marketplace.

 

[starcentral]

Mitch is the man: http://blog.abelcine.com/2012/11/01/sonys-pmw-f5-and-f55-defining-cfa/

Both the F5 and the F55 use Bayer Pattern CFAs, however the color dyes used to make the filter are different. The precise coloring of the dyes utilized helps define the range of colors the sensor can capture......


......The F5 CFA uses dyes for a standard REC709 (ITU-709) broadcast Color Space. The F55 CFA uses dyes for an extended Color Gamut beyond what broadcast monitors can reproduce. In fact, Sony lays claim to the greatest-ranging CFA in existence, with a Color Space beyond the range of any other technology, even film. This extended Color Gamut is what is shared between the F55 and the F65.

The article further explains why the filters were purposely designed to be different between the F5 and F55. It's worth a read.

 

[Jason Davenport]

And this discussion brought to you by: RIP
http://www.imaging-resource.com/new...ted-bayer-filter-for-digital-cameras-has-died

 

[Kolor-Pikker]

The thing is, when people hear 2160p they only think resolution has only doubled from 1080p, when in fact it has quadrupled; "4K" sounds more like it has quadrupled over 1080p.

Now then, there is 8K, which is 33 Megapixels even in 16:9 aspect ratio. Its actually 16 times the resolution of 1080p and 4 times of 4K (8.3MP at 16:9). The gap between 4K and 8K is much bigger than it sounds as well. Whenever they finally start releasing 8K TVs, they'll need to think of another more marketable name as currently both 4K & 8K are bundled in 'Ultra HD".

Actually, 4K is only double the resolution from 1080p, and it's marketing that makes it seem like it's 4x as much. People often forget that humans are only sensitive to resolution increase on a linear scale (particularly in the vertical domain), but a doubling of linear size in both dimensions requires four times the area (think inverse-square law).
It's because of this that higher and higher resolutions see diminishing returns in detail (if we ignore our capacity to see that much detail), so going from 1920x1080 to 3840x2160 we have twice the linear res, but 4x the required bandwidth, going from 4K to 8K we will again see a perceived resolution increase of 2x, but it will require 4x as much bandwidth again, or 16x as much data as 1080p.

So, for example, the resolution increase that the Dragon sensor will give the Epic of 6K over 5K is a mere +17% linear resolution, which will be hardly visible except on close inspection (if we completely disregard other advancements the new sensor may bring).

And this discussion brought to you by: RIP
http://www.imaging-resource.com/new...ted-bayer-filter-for-digital-cameras-has-died

Sad to hear that; his tech revolutionized all modern imaging technology and has yet to be surpassed by anything.

 

[vanvideo]

And this discussion brought to you by: RIP
http://www.imaging-resource.com/new...ted-bayer-filter-for-digital-cameras-has-died

Sad to hear.

Ironic, Bayer's invention with Kodak would help bring about the company's slow demise.

 

[Kolor-Pikker]

Sad to hear.

Ironic, Bayer's invention with Kodak would help bring about the company's slow demise.

Indeed, Kodak was the first manufacturer of digital cameras in the world, their first camera in 1975 had a CCD that shot a 100x100 pixel image that took 23 seconds to write to a cassette tape. Obviously it was a mere novelty at the time, but only 6 years later Sony already had a camera that shot 50 frames at TV resolution (essentially it was a video camera that shot stills ala RED). People take what we have today very much for granted.

 

[Duke M.]

Keeping in mind that Q67 is just another bayer pattern, we seem to be dealing with conjecture, speculation and the interpretation of semantics more than facts.

I look forward to some footage, but with out the XAVC plug in we couldn't even look at it.

 

[starcentral]

This is somewhat of a repost from another thread, but if Mitch's article holds any merit (which I'm sure it does) then in fact the sensors in the F5 and F55 are just regular bayer pattern sensors and thus we should expect "regular" bayer pattern resolution yields. The only similarity the F55 shares with the F65 has nothing to do with the sensor at all - only the filter itself and more specifically the color dyes used to make it.

The Sony press release indicated that the F55 "delivers color" similar to the F65 which can be misleading depending on how you read it so Mitch's article helps put everything in proper perspective.

In basic terms the F5 and F55 appear very much to be bayer pattern sensors but the filter used on the F5 is meant for REC709 color space while the F55 is meant for an extended color gamut beyond what even broadcast monitors can display. The article says Sony claims no other filter in the world can produce this level of color.

 

[Kolor-Pikker]

This is somewhat of a repost from another thread, but if Mitch's article holds any merit (which I'm sure it does) then in fact the sensors in the F5 and F55 are just regular bayer pattern sensors and thus we should expect "regular" bayer pattern resolution yields. The only similarity the F55 shares with the F65 has nothing to do with the sensor at all - only the filter itself and more specifically the color dyes used to make it.

That's very likely, the sensor is already a global shutter design, so additionally making it alternate to a standard bayer pattern is a little too cutting edge for a new product in a niche market.

 

[starcentral]

Lots of color going on here!

Lots of color going on here!

Expanding a little more on the extended color gamut of the F55 camera, we know the F55 is equal to the color space of the F65 and both those cameras exceed the color space of film. Previously the F35 was about equal to film in color space so these new filters by Sony are quite a breakthrough - but in the end what is the relevance of all this "wide" color space?

The increase in color space allows for more headroom in color grading and Sony says makes it ideal for meeting IIF ACES workflows. If you don't know what IIF ACES is all about Mike Post has written a great article with a great introduction on ACES and what its intended purpose is.

The short of it is that IIF ACES was developed by the Academy of Motion Picture Arts & Sciences technology committee, and is basically intended to provide a way to achieve more accurate color (and much more range to manipulate that color) from all cameras (both electronic and film) in a simpler way, along with being able to exchange that information among many vendors during the post production process while retaining consistency and accuracy.

Needless to say the F55 or F65 works in color space far outside of REC709 - which is the targeted color space for most (if not all) broadcast displays.

 

[Kolor-Pikker]

Needless to say the F55 or F65 works in color space far outside of REC709 - which is the targeted color space for most (if not all) broadcast displays.

Which may or may not be a good thing, I've worked as a pro photographer long enough to know quite a bit about color spaces, and the short of it is this:
Color values are always stored as a fixed value from 0-255 in 8-bit, up to 0-65,535 in 16-bit, but the color space dictates which actual colors are assigned to those values. The colors in a large color space are "spaced" further apart from each other because they have to encompass more colors within the quantization they're given.

Because of the above fact, larger color spaces need to be recorded at higher bit depths to prevent artifacts such as banding and posterization from coming into the picture, but is it really worth it? Most display devices today use 8/10-bit panels and roughly encompass the sRGB color space, with some editing monitors by NEC and Eizo being capable of rendering aRGB, and nothing can render ProPhoto except for high end 10-color printers with expensive paper... and film, which makes me think the main purpose of a film-like gamut is to be just for that - filmout.

A wide color space rendered on a device that can't reproduce the extra information isn't like using a 4K master to produce in HD, because down-sampling provides a real-world benefit of super-sampling over shooting direct-to-HD, but with color spaces that extra information is simply thrown away. If the F55's color gamut really is that big, it just can't be recorded at 8-bit because viewing the footage on a monitor will truncate the color data, resulting in an image that will have a much lower "equivalent" bit depth (as it will only show those bits that actually have colors it can show). Color space conversion is often alleviated with perceptive compression, but it's still no better than just shooting rec.709 in the first place.

TL;DR: If you don't know why you need it, a large color space is a trade off that will work against you.

 

[olliehm]

Which may or may not be a good thing, I've worked as a pro photographer long enough to know quite a bit about color spaces, and the short of it is this:
Color values are always stored as a fixed value from 0-255 in 8-bit, up to 0-65,535 in 16-bit, but the color space dictates which actual colors are assigned to those values. The colors in a large color space are "spaced" further apart from each other because they have to encompass more colors within the quantization they're given.

Because of the above fact, larger color spaces need to be recorded at higher bit depths to prevent artifacts such as banding and posterization from coming into the picture, but is it really worth it? Most display devices today use 8/10-bit panels and roughly encompass the sRGB color space, with some editing monitors by NEC and Eizo being capable of rendering aRGB, and nothing can render ProPhoto except for high end 10-color printers with expensive paper... and film, which makes me think the main purpose of a film-like gamut is to be just for that - filmout.

A wide color space rendered on a device that can't reproduce the extra information isn't like using a 4K master to produce in HD, because down-sampling provides a real-world benefit of super-sampling over shooting direct-to-HD, but with color spaces that extra information is simply thrown away. If the F55's color gamut really is that big, it just can't be recorded at 8-bit because viewing the footage on a monitor will truncate the color data, resulting in an image that will have a much lower "equivalent" bit depth (as it will only show those bits that actually have colors it can show). Color space conversion is often alleviated with perceptive compression, but it's still no better than just shooting rec.709 in the first place.

TL;DR: If you don't know why you need it, a large color space is a trade off that will work against you.

While I understand what you are getting at, its surely always better to be future proofing to a certain extent, no? Eventually the rest will catch up and monitors etc will be able to display colour gamuts that wide?

 

[Kolor-Pikker]

Future-proofing? Hmm, I really can't say, many of the color spaces and standards we use today have already been around for a very long time, and it's likely we'll still have them for a while yet. Until some manufacturers begin to produce color-accurate wide gamut screens in TVs that the average person can buy, we will have to do with what we have, most people don't even calibrate their TVs...

 

[starcentral]

While I understand what you are getting at, its surely always better to be future proofing to a certain extent, no? Eventually the rest will catch up and monitors etc will be able to display colour gamuts that wide?

It's not so much only the displays that are the issue, what Kolor-Pikker is further suggesting are the limitations of the recording format itself. i.e. 8-bit 422 50mbps recording mode on the F55 might not be enough bit-depth-resolution to adequately hold such a wide range of tonal information.

Here's a simplistic analogy:

For example imagine a black and white photo taken with a sensor capable of 14 stops. When the image hitting the sensor is processed internally by the A/D and then stored at 10-bits we get about 73 gradations per stop versus only 17-18 gradations per stop in 8-bit. With 16-bits we get 4640 gradations per stop!

It becomes quite easy to see why 16-bit resolution gives us so much more headroom in post to manipulate colours and grades, and if you've ever played around with a RED RAW file you know exactly how much latitude you have when bending at around the image. I recently posted a link to some RED examples and the variation results are mind blowing.

So we can agree that 1.) it means the "more" range you start off with and put through internal electronic processing the "less" quantization you will have per stop, and 2.) you will have even "less" quantization available per stop when dealing with lower bit-depths.

Most of us already know that 10-bit on a 13.5 stop camera like the F3 in SLOG mode gives enough quantizations to keep gradations smooth between stops for moderate to extensive grading, but what "extended color gamut" of the F35, F55, and F65 could mean is you may very well want or need to shoot in bit rates much higher (ie. 16-bit RAW mode only).

Knowing Sony they've got our ass covered and are doing some heavy internal down-sampling to get the color space within the lower internal 8-bit/10-bit codecs... but Kolor-Pikkers' recent post sure has me wondering......

 

[Mitch Gross]

I can confirm that both the F5 and F55 utilize traditional Bayer pattern CFAs and traditional Bayer pattern orientation.