F5/F55 Sensor resolution and bayer pattern

[coltonokeefe]

It shouldn't be any different from shooting RAW on any other camera (with OLPF) but of course further "tweaking" will depend on what kind of tools are made available by whom, and what can be done in them, and whether they are hardware or software based.

For example the Red Rocket card does realtime 4k debayering but I don't believe allows you to chose a debayering interpolation method or algorithm, nor allow any selectable OLPF compensation or noise reduction in the process. Whereas if you drop a RED Raw clip into Adobe Premiere you can go into the source settings for the clip and adjust some small level of debayering detail, chorma denoiser, and OLPF compensation.

You are right however most of the resolution/aliasing tradeoff happens with the OLPF but the thought of having some control over the complexity level of the interpolation algorithm based on your computing power will give you some power to further tweak your image. But these tweaks are minor and may only come in useful on a handful of clips.

Interestingly enough the BMCC doesn't use a OLPF at all.

Very interesting concept. I had never considered a manufacturer giving us multiple debayering algorithims to choose from. It's highly unlikely that someone would offer that just yet, but would be very intersting to see this as a spec'd feature at somepoint in the future. Just about every camera on the market gives users the option of selecting a certain "flavour" of codec depending on their needs. Selectable Raw debayering alorithims seems like the next logical step in the progress of Raw video, provided it's actually possible.

 

[starcentral]

These two statements contradict each other. As the first says horizontal pixel count * 0.4 = actual resolution. The second says that horizontal pixel count * 0.83 = actual resolution.

The first statement is probably derived from Nyquist theory which states that you need to have a sampling rate = to at least twice the maximum frequency to be measured. But what do "sampling rate" and "frequency" mean in the context of digital imaging? The long and the short of it is that these terms are somewhat misused. So in fact a 4K sensor could certainly resolve 4K of resolution (but of course not more than 4K).

But how is this possible, as Red is saying you need a 5K Epic to resolve 4K? That's b/c the Epic's sensor uses a Bayer mask, which reduces resolution. The rule of thumb is that using a Bayer pattern reduces resolution by ~ 0.8. This is also why the Monochrome Epic has significantly higher resolution, as it has no Bayer mask.

Bobby those statements were meant as examples to show that no matter what you can not end up with more than you start off with, there is always some "loss". A 4k sensor therefore (in the sense of it measuring approximately 4000x3000) will not deliver 4000 lines of resolution without serious aliasing issues which is why after OLPF and debayering you can expect .83 of resolvable lines of resolution from it.

You still have a few errors in those statements. The F65 is a 17.6MP sensor to produce an 8k wide image in the future. Yes it produces 4000 lines of resolutoin now, but 17.6 MP isn't required for 4k resolution.

The total MP is only a rough estimate of resolution. OPLF, CFA, photosite pattern, photosite size all have an impact. Even alignment of the camera can have an impact if the photosites are in a straight grid.

Duke, those are not my statements directly, they were according to Steve's. So we need to ask him.



In the end, my guess is that the F5 and F55 will yield the same resolvable lines of resolution but the F55 if in fact it is using Sony's new Q67 filter pattern will only be better in the sense that it will not produce any color artifacting traditional with bayer pattern sensors.

 

[optitek]

I real life it's about 65-70%. And I have the Optimator to prove it. I've never seen any camera other than the F65 resolve more than 2500, 2600 lines.
The F65 gets close- maybe 3800 or so....

 

[Steve Castle]

This discussion is on the resolution of these sensors, so what is the general consensus so far?

If the F55 uses the same Q67 pattern layout as the F65 (a 17.6MP sensor delivering approx ~4000 lines of resolution) what will the F55 with its 8.9MP Q67 sensor deliver?

Steve you had said that the Q67 17.6MP sensor will deliver a 8.9MP image, so since the F55 is only 11.6MP/8.9MP what do you reckon it will deliver?

By the way the F35 was mentioned somewhere but note the F35 uses vertical strips of RGB filters so although it is a 12MP sensor compared to a bayer pattern sensor with approximately the same MP it delivers significantly less resolution. ie. the F35 delivers approx 1.9k lines of resolution while the original RED ONE with its 9.5MP bayer pattern sensor delivered approximately 3.2k lines - a whopping 1.3k difference.

I think that most people are used to thinking of image sensors in the bayer mentality. That one photosite is one pixel. A 10MP sensor makes a 10MP image. Bayer's popularity has always been because it inflated resolution even at a penalty in image quality, marketing "Megapixels" was easy, for stills more megapixels always meant better; quantity over quality.

However for video, where resolution is fixed, more megapixels haven't meant much as the delivery format doesn't change, that is, until RED entered the scene. More megapixels, more "K" matters in marketing these cameras now. Yet how we measure the penalty to resolution has remained constant over vastly different sensor types with different CFAs; F35, Canon C500, F55/F65, etc. For the F35 and the new Canon C cameras, obviously there is very little correlation between the number photosites and the effective resolution. This is because purpose of the sensor isn't designed to produce a very high resolution image, rather the image it can within the limitations of the delivery format. When discussing different types of sensors we cannot make a direct correlation of how it will perform on a resolution chart.

So, the F55 & F65 are both 4K cameras. One has a 20MP sensor, the other 11.6MP, when just the number of photosites are calculated, but both cameras in the end will deliver a 4K image. The F65 will produce a Q67 image at 4K, meaning if has 67% of the color data, where the green pixel data is straight off the sensor, and red and blue can be figured via simple interpolation. This way, the F65 can get as close to 4:4:4 as possible without sacrificing too much in pixel density in what may be an effort in diminishing returns. The F55 is only around ~Q43 or so at 4K (For comparison a classic bayer would be Q33 at whatever its native resolution is). The F55 still uses the same CFA, but it requires a lot more guesswork and it shouldn't be designed to achieve a true 4:4:4 image even though the file itself may support it. The difference is quality over quantity.

The F65 also can produce 6K and 8K files as well. Resolution will increase, but again, 8K in particular is achieved through vertical interpolation of the sensor. These "Q67 double bayers" don't use classical demosaicing algorithms to make a final image. To think of it another way, the F65 takes a lot less interpolation to make a 4K image than the F55. So for the F55/F65 there is a lot less guess work in forming an image compared to a traditional bayer (which comes in the penalty of having a much lower effective output resolution relative to your sensor resolution). For this reason, I would suspect that the F55 should achieve fairly close to its effective resolution which is 8.8MP (more so than normal bayer).

 

[bruceallen]

This thread confuses me. Sony advertises the F55 and F5 both as 8.9 megapixels effective. Pretty sure that means they're both Bayer.

Yes, the sensors have some extra area beyond that - but I think that area is used for stuff like fixing black shading issues?

My understanding was something like they cover part of the sensor completely (so it should be black) and read it out - what they get is black + sensor noise.

This gives Sony a way of gauging sensor noise so that you don't have to do the damn RED black shading dance when sensor temperature or exposure times change drastically.

Very interesting concept. I had never considered a manufacturer giving us multiple debayering algorithims to choose from. It's highly unlikely that someone would offer that just yet, but would be very intersting to see this as a spec'd feature at somepoint in the future. Just about every camera on the market gives users the option of selecting a certain "flavour" of codec depending on their needs. Selectable Raw debayering alorithims seems like the next logical step in the progress of Raw video, provided it's actually possible.

Actually, Slumdog Millionaire, which shot on the Silicon Imaging camera (which did compressed raw before RED did) used different Cineform debayer settings for different scenes:
http://techblog.cineform.com/?p=2397
>
Sidebar About Using Different DeBayer Algorithms. When discussing workflow with the post team that worked on Slumdog Millionaire, we discovered they used different CineForm DeBayer settings depending on varying sharpness of the source material. Recall in Slumdog that the early part of the movie has a lot of running through the slums of Mumbai. Not all scenes were perfectly in focus. The post team experimented with varying sharpening strategies including post-sharpening after DeBayer. They ultimately determined that sharpness applied using different CineForm DeBayer algorithms provided superior results than post sharpening. You might experiment with this on your own projects. End Sidebar….
>

Bruce Allen
www.boacinema.com

 

[starcentral]

This thread confuses me. Sony advertises the F55 and F5 both as 8.9 megapixels effective. Pretty sure that means they're both Bayer.

The F55 apparently uses the "new" Q67 filter pattern but again its still only a 4096x2160 sensor so doubtfully will deliver true 4k and likely be closer to 3.2k.

Actually, Slumdog Millionaire, which shot on the Silicon Imaging camera (which did compressed raw before RED did) used different Cineform debayer settings for different scenes:
http://techblog.cineform.com/?p=2397
>
Sidebar About Using Different DeBayer Algorithms. When discussing workflow with the post team that worked on Slumdog Millionaire, we discovered they used different CineForm DeBayer settings depending on varying sharpness of the source material. Recall in Slumdog that the early part of the movie has a lot of running through the slums of Mumbai. Not all scenes were perfectly in focus. The post team experimented with varying sharpening strategies including post-sharpening after DeBayer. They ultimately determined that sharpness applied using different CineForm DeBayer algorithms provided superior results than post sharpening. You might experiment with this on your own projects. End Sidebar….
>

Bruce Allen
www.boacinema.com

Great link Bruce, RAW certainly holds a lot of possibilities after the fact which makes it ideal for higher end productions, or simply just ones that plan to undergo further "polish" in post.

 

[Steve Castle]

The F55 apparently uses the "new" Q67 filter pattern but again its still only a 4096x2160 sensor so doubtfully will deliver true 4k and likely be closer to 3.2k.

I really don't think you understand Q67. You don't add a normal bayer penalty ON TOP of the Q67. Again, Q67 uses a panchromatic pixel that takes straight green pixel data, and half blue/red, it has 67% of RGB data (normal bayer has 33%). Which is why you need a higher resolution sensor to make a 4K image (in this case 11.6MP). You don't debayer the final effective resolution of the sensor in this case. So your calculation are penalizing the sensor twice.

 

[starcentral]

I really don't think you understand Q67. You don't add a normal bayer penalty ON TOP of the Q67. Again, Q67 uses a panchromatic pixel that takes straight green pixel data, and half blue/red, it has 67% of RGB data (normal bayer has 33%). Which is why you need a higher resolution sensor to make a 4K image (in this case 11.6MP). You don't debayer the final effective resolution of the sensor in this case. So your calculation are penalizing the sensor twice.

So you are insistent that the F55 will deliver the same resolution as the F65 simply because it is Q67?

 

[Steve Castle]

So you are insistent that the F55 will deliver the same resolution as the F65 simply because it is Q67?

I've explained this in detail a few posts above.

But I'll summarize again, the F65's goal is to achieve what Sony calls "true 4K", which is to get 16-bit 4:4:4 4K. Sony's logic is to get 67% of RGB data, and do zero interpolation of green pixels, and to only interpolate blue/red pixels (otherwise it would need 26.4MP and smaller photosites to get 100% RGB data at 4K). The F55 will not achieve that, closer to Q43, while it'll still be 4K it will only achieve 4:2:2 even after interpolation; which may be good enough for most people.



EDIT: Sony has a whitepaper with pictures to aid in understanding the difference between F65 and a classic bayer:

Direct Link:
http://pro.sony.com/bbsccms/assets/files/show/highend/includes/F65_Camera_CameraPDF.pdf

 

[bruceallen]

So - you guys are sure about this Q67 thing?

1. Sony not mentioning it in the the press releases is odd.
2. Doing an 8.9 megapixel Q67 for the F55 and an 8.9 megapixel Bayer for the F5 is odd too.

I also don't buy for a second the idea that the F55 has a Q67 sensor with 11 or more megapixels. Because then Sony would have marketed it as such.

They market the F65 as a 20 megapixel sensor - eg they count every sensor site, so I'd expect them to do the same with any other Q67 product.

"F5 has 8.9 megapixels. F55 has a 16 megapixel or 11 megapixel or whatever Q67 global shutter sensor"

I still suspect that folks have misread the "F65 CFA" bit as meaning Q67 when in fact it refers to the purity of the color filters, not their arrangement.

Sony has stated that the F5 has the (less pure) red, green and blue filters from the F3, F55 has the (very pure wide-gamut) red, green and blue filters from the F65.

Bruce Allen
www.boacinema.com

 

[starcentral]

I don't believe the F55 is Q67 or it would be widely publicized. Also the F55 would be advertised as a 17.6MP sensor, or 8k sensor, or some other variation of the "true" pixel count.

Great white paper by Sony. The second and last sentence of the first slide says exactly what I had been saying from the beginning;

So a 4k sensor might have 4096 photo sites on every row. Unfortunately, the actual resolution is less because these sensors use Bayer color filter arrays....

....However sophisticated the de-bayering algorithm may be, they still can not recreate that information which was never captured to begin with. For this reason, the final resolution fails, to various degrees, short of true 4k.

Anyway if the F55 is Q67 then even better for all of us no doubt!

 

[Steve Castle]

I don't believe the F55 is Q67 or it would be widely publicized. Also the F55 would be advertised as a 17.6MP sensor, or 8k sensor, or some other variation of the "true" pixel count.

Great white paper by Sony. The second and last sentence of the first slide says exactly what I had been saying from the beginning;



Anyway if the F55 is Q67 then even better for all of us no doubt!

There are two likely possibilities here:

1.) This, as Alister Chapman and others say, is a double-bayer Q67 sensor. Keep in mind, a double-bayer doesn't need to have 17.6MP to be 4K. It just wouldn't be 4:4:4. It would require a lot more interpolation than the F65. But its more than possible.

Sony's marketing department tends to be very conservative, or less capable, depending on your perspective. They don't shout things off the rooftops like Red does. They don't call the F65 the "8K Super-Epic Dragon" for instance. They also tend to be more opaque when their specs are concerned. Which can lead to these sort of confusions.

2.) Its a 4K (8.8MP) image supersampled from a 11.6MP(~4.7K) bayer. The gross pixel count on the sensor is 11.6MPs. We know this. We know the F55 is also global shutter, meaning it doesn't skip lines, etc, as it takes the entire frame. Its effective megapixel is just 8.8MP. The delta between gross pixels and effective pixels is VERY large for a bayer sensor; sensors do need extra space for calibration and other purposes, but not this much. As an example, the Sony's A99 sensor (the most recent Sony bayer sensor), the gross pixel is 24.7MPs, its effective pixel is 24.3MP; which is a 1.6% difference. In the F55 nearly a quarter (24%) of its gross pixels would be unused to make the final image if it wasn't supersampled and was using a classic bayer CFA.

Sony may be taking the F3 route for marketing. The F3 had a 3.34MP sensor, but it "effectively" had a 2MP 1080p sensor (as that is the only output you could get out of the camera). So for the F5 & F55 Sony may be using a 11.6MP bayer, and outputing 8.8MP(4K), supersampling to compensate for the lost resolution that you would lose in debayering. This would make the F5/F55 have a around a 4.7K sensor.

 

[Duke M.]

Sony may be taking the F3 route for marketing. The F3 had a 3.34MP sensor, but it "effectively" had a 2MP 1080p sensor (as that is the only output you could get out of the camera). So for the F5 & F55 Sony may be using a 11.6MP bayer, and outputing 8.8MP(4K), supersampling to compensate for the lost resolution that you would lose in debayering. This would make the F5/F55 have a around a 4.7K sensor.

The specs that B&H posted says its 8.9MP and only 8.8 are necessary for 4k, so the possibility of oversampling at 1% is there. Its been stated that part of the overage between the 11.6MP sensor and the effective 8.9MP area is at least partially being used for an automatic black balance and to read the sensor noise for noise removal.

Since a Q67 bayer filter is 100% of green, and half of red and blue a very accurate 4.2.2 color can be created. Twice as high on red and blue as a regular bayer filter.

Really though the proof is going to be in the actual pictures. I can't wait to do some pixel peeping.

 

[Steve Castle]

The specs that B&H posted says its 8.9MP and only 8.8 are necessary for 4k, so the possibility of oversampling at 1% is there. Its been stated that part of the overage between the 11.6MP sensor and the effective 8.9MP area is at least partially being used for an automatic black balance and to read the sensor noise for noise removal.

Since a Q67 bayer filter is 100% of green, and half of red and blue a very accurate 4.2.2 color can be created. Twice as high on red and blue as a regular bayer filter.

Really though the proof is going to be in the actual pictures. I can't wait to do some pixel peeping.

According to the press release the sensor is 11.6MP producing a 4096 x 2160 image (8.85MP). So oversampling is around 24% not 1%. You don't need 24% of the sensor to do things like black balance.

http://www.sony.co.uk/pro/article/pr-thenewf-new-4k-cameras

Press release.

Basingstoke, 30 October 2012 – Sony is unveiling two new CineAlta™ 4K cameras, complemented by a full line-up of new accessories. The new cameras, the PMW-F55 and PMW-F5, fill a critical gap in large-sensor acquisition between Sony's PMW-F3 camcorder and its F65 CineAlta Premium 4K camera, giving content creators new levels of flexibility and creative options for HD, 2K or 4K acquisition and production.

With a modular and compact design for easy configuration, each camera features a new type of 4K Super 35mm image sensor with a 4096 x 2160 resolution (11.6M total pixels).

The 20 megapixel F65 remains the flagship of Sony’s large sensor camera family. The 8K sensor-equipped F65 continues to evolve, with new features designed for the most demanding high-end imaging applications. These features include a new high frame rate mode for 4K RAW recording at 120P to deliver stunning 4K images with super smooth slow motion capabilities, and image capture using the SR codec at 12-bits. Future updates to the F65 will deliver 6K de-mosaicing, along with other major new features.

 

[bruceallen]

Quick thought: is the FS700 oversampling by 24% as well? Sony also states that it has an 11.6 megapixel sensor... of which 8.3 of those megapixels are being used to create the HD image.

BTW has anyone figured out whether the FS700 is Bayer? Q67? Something else?

I still suspect that both the F5 and F55 are boring old Bayer sensors that produce 4K without oversampling - eg they just take the center 8.9 megapixels. What the other 1.7 megapixels are used for is anyone's guess. Maybe it's a 4:3 sensor and they'll unleash an anamorphic version Again, I suspect that the outer stuff is used somehow for whatever black shading / fixed pattern noise reduction technique they use. But I have no idea

But boy... if the F55 or F5 were indeed sampling the full 11.6 megapixels, that would be awesome. I fear not though. Especially since then the raw image would be a higher resolution than 4096x2160. Then raw resolution would be 4352x2662.

OK, yeah. My bets are on the table:

BET 1: It is not oversampling. It is only using the middle 8.9 megapixels.
BET 2: There is no difference in color filter pattern between the F55 and F5 - just the F55 uses the F65's purer color filters. This has gotten everyone confused. Whatever pattern (Bayer, Q67, etc) is used in the FS700 is also used in the F5 and the F55.

Bruce Allen
www.boacinema.com

 

[nyvz]

Quick thought: is the FS700 oversampling by 24% as well? Sony also states that it has an 11.6 megapixel sensor... of which 8.3 of those megapixels are being used to create the HD image.

BTW has anyone figured out whether the FS700 is Bayer? Q67? Something else?

I still suspect that both the F5 and F55 are boring old Bayer sensors that produce 4K without oversampling - eg they just take the center 8.9 megapixels. What the other 1.7 megapixels are used for is anyone's guess. Maybe it's a 4:3 sensor and they'll unleash an anamorphic version Again, I suspect that the outer stuff is used somehow for whatever black shading / fixed pattern noise reduction technique they use. But I have no idea

But boy... if the F55 or F5 were indeed sampling the full 11.6 megapixels, that would be awesome. I fear not though. Especially since then the raw image would be a higher resolution than 4096x2160. Then raw resolution would be 4352x2662.

OK, yeah. My bets are on the table:

BET 1: It is not oversampling. It is only using the middle 8.9 megapixels.
BET 2: There is no difference in color filter pattern between the F55 and F5 - just the F55 uses the F65's purer color filters. This has gotten everyone confused. Whatever pattern (Bayer, Q67, etc) is used in the FS700 is also used in the F5 and the F55.

Bruce Allen
www.boacinema.com

I also bet its just 8.9MP, normal old 4k RAW bayer and 11MP is just some irrelevant spec of the sensor in some other aspect ratio or with some extra unused sensor area. If I recall there are also specs for FS700 that say 11MP but 8.9MP active/effective.

Even if the F55 is Q67, its probably just 4k Q67 which is just 4k bayer with a slightly rotated orientation... It's probably not anyway though.

 

[Steve Castle]

Quick thought: is the FS700 oversampling by 24% as well? Sony also states that it has an 11.6 megapixel sensor... of which 8.3 of those megapixels are being used to create the HD image.

BTW has anyone figured out whether the FS700 is Bayer? Q67? Something else?

I still suspect that both the F5 and F55 are boring old Bayer sensors that produce 4K without oversampling - eg they just take the center 8.9 megapixels. What the other 1.7 megapixels are used for is anyone's guess. Maybe it's a 4:3 sensor and they'll unleash an anamorphic version Again, I suspect that the outer stuff is used somehow for whatever black shading / fixed pattern noise reduction technique they use. But I have no idea

But boy... if the F55 or F5 were indeed sampling the full 11.6 megapixels, that would be awesome. I fear not though. Especially since then the raw image would be a higher resolution than 4096x2160. Then raw resolution would be 4352x2662.

OK, yeah. My bets are on the table:

BET 1: It is not oversampling. It is only using the middle 8.9 megapixels.
BET 2: There is no difference in color filter pattern between the F55 and F5 - just the F55 uses the F65's purer color filters. This has gotten everyone confused. Whatever pattern (Bayer, Q67, etc) is used in the FS700 is also used in the F5 and the F55.

Bruce Allen
www.boacinema.com

These sensors are S35 and not a 4:3. A crop would be pointless in a dedicated S35 sensor, and would mean a noticeable crop factor from a standard S35. Let's put it this way, we have 2.75MP wasted, that delta is larger than the resolution of 1080P. As an engineer you would want to either fill the area and get larger photosites or, as mentioned, supersample it. If its just regular old bayer you also don't want to have exactly the target output resolution as you lose a considerable amount of data during debayering. You would want to oversample, this has been common practice for even low-priced 1080p CMOS camcorder (which usually had a ~3.2MP sensor to make a 1080p image).

As for the FS700, we have no clue as it currently only records at 1080p only. To do 4K it seems you need a pricey recording unit, meaning a supersampled 4K would certainly be justified given the price you would need to pay. But really, recording at 11.6MP isn't that much of a technical leap over 8.85MP these days; especially given the price point and the fact that these sensors are clearly very fast. This isn't some dSLR we are talking about.

Given the price that we are expecting, and given that Sony is one of the most aggressive sensor manufacturers in the world right now, I would guess that if it is a bayer, its supersampled from a higher resolution. This is obviously true for the global shutter, which takes the entire frame, and is also clearly a different sensor than that of the F5 & FS700.

 

[Duke M.]

I think we're all a bit right and only time will tell. The published specs do say an 11.6MP sensor with 8.9MP effective. It might be a 4:3 aspect ratio sensor, but THAT is a lot of wasted real estate. Beyond black balance they may be using it for another proprietary reason since its a new sensor series.

Whether its only the Q67 dies that were used in the F55 for pure color we won't know for awhile, but what was said was that the F55 has the F65's color filter ARRAY. That would seem to indicate that its Q67. Also, the global shutter on the F55 tells you that its not the same sensor as the F5 since the global shutter requires a driver for each pixel. That's hardware, not software.

 

[nyvz]

These are 4k raw cameras. Supersampling works for debayered outputs and implies downscaling. It is very difficult to scale RAW bayer data, binning is possible but requires a sensor with 4x the resolution. That said it is not impossible that RAW scaling could be done in some new proprietary way, since it seems the F65 does it in some way by offering 4k and 8k raw recording from its effectively 6k sensor. But the likelihood that sony is scaling 4.5k bayer data down to 4k is incredibly low. More likely such an operation would introduce more artifacts than it would increase resolution.

 

[starcentral]

If in fact the F55 has a rotated photo sites by 45 degrees then I have a new calculation for the resolvable lines of resolution for the F55 to be about 3700 lines, whereas the F5 will still be around 3200 lines.

By the way did anyone notice the aspect ratio for the new F's are closer to 1.85 as opposed to 16:9 as found on F3, Fs700, Fs100, etc.. ?

So shooting 16:9 you will chop off 256 pixels horizontally (left and right sides) to get it more like 3840 (h) x 2160 (v)