New Blackmagic Camera July 16th?

[SJX]

Yeah. I mean in my experience Monsto doesn't have more dynamic range than Alexa, in the way I like to shoot and test anyway. And yet on RED's site, it's a 17+ stop camera.
https://www.red.com/dsmc2

Arri ? they say 14 stops on their website.

https://www.arri.com/en/learn-help/technology/image-processing

There's a MASSIVE difference between 14 and 17 stops....Anyone else think those numbers are right ?

This is why I detest using "stops" as a way of measuring dynamic range. They're always a BS number unverifiable and totally subjective. Complete with non-sensical unscientific phrases like "usable" to "bankable" stops

JB

And blackmagic says Ursa G2 has 15 stops but cinema 5D gives it 12.1 or 12.5 stops. Everybody lies. Arri is the only one which publishes the DR close to what it really is. This ursa 12K will probably be 12 stops or maybe less than G2. Looking forward to the cinema 5D tests.

 

[Eric Coughlin]

And blackmagic says Ursa G2 has 15 stops but cinema 5D gives it 12.1 or 12.5 stops. Everybody lies. Arri is the only one which publishes the DR close to what it really is. This ursa 12K will probably be 12 stops or maybe less than G2. Looking forward to the cinema 5D tests.

I wish Cinema5D would stop doing their bogus dynamic range tests so that people don't continue getting misinformed by their highly flawed tests, but journalistic sensationalism drives traffic and business so who's to stop them.

 

[AndreeOnline]

I wish Cinema5D would stop doing their bogus dynamic range tests so that people don't continue getting misinformed by their highly flawed tests, but journalistic sensationalism drives traffic and business so who's to stop them.

You‘ve written something similar a couple of times, Eric. Would you mind sharing what you think the flaws are, instead of just saying they are there?

Like John says above, there are many ways to measure DR – many of them a bit willy nilly. Cinema5D‘s tests seem pretty repeatable to me and I don‘t see a huge problem.

Cinema5D‘s testers are pretty clear about how they test and where the numbers come from. I like that there is a noise cutoff and that the numbers are generated by Imatest, which seems more unbiased to me than a human that might have a brand preference. This assumes that the images that are fed into Imatest are being generated the same way, but I have not seen anything that points in another direction.

As a personal observation, I also feel that the stops between 12-18 are gained more cheaply than the earlier stops. It‘s often a human digging into noisy patches, squinting with their eyes, arriving at a benevolent conclusion regarding „their“ camera. Test procedures arriving at DR ranges between 10-13 stops will show much smaller differences between high and low end cameras compared to procedures that rate between 11-19.

So, the same two cameras might end up as: 11.3 vs 12.2 compared to 13 vs 18. I much prefer the first case and think it‘s more representable of what the actual images look like.

Anyway, I‘m truly open to hearing your objections.

 

[Samuel H]

And blackmagic says Ursa G2 has 15 stops but cinema 5D gives it 12.1 or 12.5 stops. Everybody lies. Arri is the only one which publishes the DR close to what it really is. This ursa 12K will probably be 12 stops or maybe less than G2. Looking forward to the cinema 5D tests.

I'm looking forward to the LensProToGo tests, which are much more informative than a black-and-white test chart with only big shapes with no fine detail, as this doesn't say much about noise and detail in the darkest stops.

I wouldn't say it's non-informative, though: it's an upper bound to the DR of the camera (if it can't discern the big boxes, it can't discern detail, and it doesn't matter how noisy it is or how much NR it is applying there). Definitely better than marketing claims, which provide absolutely no information (even within-brand, the marketing claim doesn't tell me, or at least not with any confidence, if the UMP-12k has more DR than the UMP-G2 or than the p6k).

 

[DLD]

So, as one thread swerved into a discussion about batteries (and electric cars), this drifts into the DR tests. I think most here agree that the outer bands are always up for a debate - too noisy or too blown or too blue or whatever. The testing methodologies also seem to be accepted and, to some degree, acceptable ... but could there be an alternative method? Something less prone to the interpretation?

I know, in audio, there was an old battle of "clean watts". That more or less ended by running a signal through an oscilloscope and seeing the distortion levels on a grid. In automobiles, car's power on a dynamometer is a true measuring stick compared to a manufacturer's claim. (I mentioned how a 330 HP BMW engine may test at 340 on the dyno and how a Ford Mustang 330 may test at 260). Technology has moved forward. Shooting at a white board seems far too backward of a proposal.

What say you?

PS. I know there are different dyno's which won't show the identical results on the same car. Likewise, 0-60 depends on a particular driver. There's a very nice YouTube fellow who drove a Hyundai Veloster on Nürburgring Nordschleife in about 10 minutes. He then laughed and said that a professional driver can do 10 minutes in a minivan.

 

[Grug]

I read that it was so they didn't have to re-design/engineer the camera housing itself. I would only use it with SSDs anyway so no big deal for me.

Which would be fine, except for the fact that the SSD recorder receives its feed from the external USB-C port. Which is only vaguely less terrifying a connection than Micro HDMI.

 

[tull_power]

Which would be fine, except for the fact that the SSD recorder receives its feed from the external USB-C port. Which is only vaguely less terrifying a connection than Micro HDMI.

From the presentation video - there are screws at both ends of the cable, securing firmly the USB-C plugs to the SSD recorder and camera.......

on a seperte note - has anyone opened the sample raw files in Resolve and compared them to the Ursa Mini G2 in terms of dynamic range, noise, highlights, shadows and texture?

Very curious about what the pros on the forum think about the footage; from my part I was blown away back when the 6k was released. Most beutiful image I've ever seen - with a ton of lattitude.

Even that is beyond my means at the moment but I'm intrigued by the age we're living in!

I'm old enough to remember $$10k would buy a low end pro-video camera back in the day - and now you can get a monster recording 80 megapixels 60 times a second!

 

[Tom Roper]

Why couldn't you just adapt EF to RF?

Not saying it’s unworkable at all, it’s just not a feature that matters to me. The other poster was laying it out with excitement.

 

[Eric Coughlin]

You‘ve written something similar a couple of times, Eric. Would you mind sharing what you think the flaws are, instead of just saying they are there?

Like John says above, there are many ways to measure DR – many of them a bit willy nilly. Cinema5D‘s tests seem pretty repeatable to me and I don‘t see a huge problem.

Cinema5D‘s testers are pretty clear about how they test and where the numbers come from. I like that there is a noise cutoff and that the numbers are generated by Imatest, which seems more unbiased to me than a human that might have a brand preference. This assumes that the images that are fed into Imatest are being generated the same way, but I have not seen anything that points in another direction.

As a personal observation, I also feel that the stops between 12-18 are gained more cheaply than the earlier stops. It‘s often a human digging into noisy patches, squinting with their eyes, arriving at a benevolent conclusion regarding „their“ camera. Test procedures arriving at DR ranges between 10-13 stops will show much smaller differences between high and low end cameras compared to procedures that rate between 11-19.

So, the same two cameras might end up as: 11.3 vs 12.2 compared to 13 vs 18. I much prefer the first case and think it‘s more representable of what the actual images look like.

Anyway, I‘m truly open to hearing your objections.

I've probably stated why here before but here goes again. Well for one they rated the 5D Mark II's dynamic range in video mode as higher than the C300 Mark II. Having used both cameras the 5D always appeared to me to have significantly less dynamic range. I trust my eyes and experience more than I trust the tests of others.

And then there are their evaluation methods. They have basically pulled out of thin air what they consider acceptable noise and have failed to take into account in-camera noise reduction and compression schemes of different cameras. The 5D Mark II for example has a high compression rate (and might have in-camera noise reduction as well) which significantly reduces the fine level of noise in an image. Cinema5D, instead of thinking, "Well, this makes the overall image crap," thinks, "I don't see noise in this crappy image so this counts as "usable" dynamic range." Essentially, if they had turned up the in-camera noise reduction setting of the C300 Mark II when they did their test (or done post noise reduction, or recorded to a highly compressed crappy codec, etc.), based on their flawed testing method, they would have come up with a higher dynamic range result for the camera. Then they published a sensationalist article titled, "Canon C300 Mark II Lab Test – Dynamic Range 2 Stops Less Than Expected," shortly after the C300 Mark II came out, and unfortunately, a lot of the population was not technically savvy enough to see the flaws in their tests and that article I imagine hurt sales and business for the C300 Mark II.

Their tests are basically skewed towards giving an advantage to cameras with lower resolution, higher in-camera noise reduction, and higher compression. Which, big surprised for a website literally named after the "5D" and which has a following that I would imagine is heavily of DSLR users. DSLR users always like to hear how their $1k-3k camera is just as good as the much more expensive cameras. I recall a 5D owner wedding videographer friend texting me that article when it came out, and how it made him happy to be with his 5Ds and consider the C300 Mark II a waste of money (as many DSLR users are not informed enough to know the overall differences between DSLRs and more expensive video cameras).

It's often the case that shooting RAW on a camera can result in more noise than shooting compressed, but the noise is more controllable, easier to reduce in post, and overall RAW often helps with increasing dynamic range. Again, based on Cinema5D's method of using noise levels to determine usable dynamic range, shooting RAW would likely decrease the "usable" dynamic range based on their testing methods, which is contrary to the reality of shooting RAW.

If a camera manufacturer wanted to score really good on Cinema5D's tests, all they'd have to do is make a camera that by default (since Cinema5D doesn't tinker with camera settings or post noise reduction to get the best dynamic range results) has a very high level of noise reduction in camera, then pump out a super soft lower than 480p resolution image, and Cinema5D's test may make a headline, "iPhone12 has 17 stops of dynamic range at 240 fps!" Which would be ludicrous, just like their tests are.

I just have trouble giving validity to a test which fails to take into account image quality and other factors when determining dynamic range and goes only off of noise level at default camera settings.

Unfortunately their tests still seem to be given attention because it seems they're the only popular video website around actively doing dynamic range tests, I'm guessing perhaps because other (more legitimate) sites like Newsshooter know how subjective dynamic range testing can be and don't want to publish non-factual articles that are purely based on subjective opinions. And yeah, perhaps there is something that can be learned from their tests since they use the same methods on various cameras, but overall I think their tests have done more harm than good in spreading accurate information to the film community which is why I often feel the need to put them down, and don't always feel like re-typing my reasoning in each new post on the matter.

 

[Tom Roper]

They have basically pulled out of thin air what they consider acceptable noise and have failed to take into account in-camera noise reduction and compression schemes of different cameras. The 5D Mark II for example has a high compression rate (and might have in-camera noise reduction as well) which significantly reduces the fine level of noise in an image. Cinema5D, instead of thinking, "Well, this makes the overall image crap," thinks, "I don't see noise in this crappy image so this counts as "usable" dynamic range." Essentially, if they had turned up the in-camera noise reduction setting of the C300 Mark II when they did their test (or done post noise reduction, or recorded to a highly compressed crappy codec, etc.), based on their flawed testing method, they would have come up with a much higher dynamic range result for the camera.

Their tests are basically skewed towards giving an advantage to cameras with lower resolution, higher in-camera noise reduction, and higher compression.

It's often the case that shooting RAW on a camera can result in more noise than shooting compressed, but the noise is more controllable, easier to reduce in post, and overall RAW often helps with increasing dynamic range. Again, based on Cinema5D's method of using noise levels to determine usable dynamic range, shooting RAW would likely decrease the "usable" dynamic range based on their testing methods, which is contrary to the reality of shooting RAW.

Andree makes great points as always. But I agree with Eric about this. This is the flaw of DR testing, even using Cinema5D's Imatest methods, there is no way to account for s/n discrepancies that originate from in-camera noise reduction schemes.

 

[Tom Roper]

There is a counter point to this. While interesting to know the DR of the sensor, isn't it the DR of the system that we should care about? Or who has the best s/n? Fine detail dispersed in waters polluted with shot noise; Is that better to have than tasteless clean drinking water?

BRAW from my pocket 6K is very clean everywhere, noise notably absent in the low key. 16 bit raw from my F55 can seem noisy all the way into the facial tones. Is it more revealing? I'm really not sure. I struggle with this in grade, and spend more time with the F55, seeking the right balance between acceptable noise and temporal/spatial NR settings in Resolve. Sometimes it feels like there is more saturation to the colors. The Resolve CIE chromacity triangle seems to be showing this. Shoot the moon Alice! F55 fills rec.2020 all the way to the RGB endpoints, the P6K fills the G most of the R but not as much of the B.

Is this why the F55 looks more like it's going to air than cinema? You don't have to push the color to be concerned of having skin break out in splotches of R, G acne if you have underexposed areas. This shows up as a problem in PQ-ST.2084-HDR. There is less tendency for this in pocket 6K, but the color is more muted in the presentation.

 

[John Brawley]

Everybody lies. Arri is the only one which publishes the DR close to what it really is.

Their number is actually at least a stop under what it really is. Even Arri lied

That's the point. There really isn't a standard way to measure DR. And the real elephant in the room is that most DR specs from a manufacturer are generally based on calculating from an idealised model of the sensor.

They do this because it means you don't have to test dozens of sensors to get a true baseline (news flash, every single sensor performs differently even on the high end) and then factor testing methodology and lenses etc.

And then the really true reason why you can't get standardisation. There's no way to quantitively measure noise. so some HUMAN BEING sits there and say, thats' too noisy, or that's acceptable noise.

And with three different noise patterns you generally get 4 different opinions of what's acceptable detail and what's not.

So they use maths. And that's the way they all do it. And that's why no-one ever can get close to what they claim. And why it's a kind of meaningless metric. Or least, one that's not that useful in the bigger picture and is more easily tested by use in the way you work.

And look at this. There were headlines screaming about RED doing 20 stops...this is a 2 generation old camera now, but here's the same chart showing 20 stops. Photographic proof it's all BS...
https://www.reduser.net/forum/showthread.php?92158-Enter-the-Dragon

So most don't sit there with a chart and calculate it, though Arri say they did with their number with a DR chart they designed but can only apparently document up to 15 stops and no-one else uses, but their number is off too...so ?


JB

 

[Amulet Man]

Their number is actually at least a stop under what it really is. Even Arri lied

That's the point. There really isn't a standard way to measure DR. And the real elephant in the room is that most DR specs from a manufacturer are generally based on calculating from an idealised model of the sensor.

They do this because it means you don't have to test dozens of sensors to get a true baseline (news flash, every single sensor performs differently even on the high end) and then factor testing methodology and lenses etc.

And then the really true reason why you can't get standardisation. There's no way to quantitively measure noise. so some HUMAN BEING sits there and say, thats' too noisy, or that's acceptable noise.

And with three different noise patterns you generally get 4 different opinions of what's acceptable detail and what's not.

So they use maths. And that's the way they all do it. And that's why no-one ever can get close to what they claim. And why it's a kind of meaningless metric. Or least, one that's not that useful in the bigger picture and is more easily tested by use in the way you work.

And look at this. There were headlines screaming about RED doing 20 stops...this is a 2 generation old camera now, but here's the same chart showing 20 stops. Photographic proof it's all BS...
https://www.reduser.net/forum/showthread.php?92158-Enter-the-Dragon

So most don't sit there with a chart and calculate it, though Arri say they did with their number with a DR chart they designed but can only apparently document up to 15 stops and no-one else uses, but their number is off too...so ?


JB

Are you saying Arri is 15+? I heard from someone who spoke with an Arri rep that they'll admit this if you press them, but they don't want to muddy the waters.

More useful is over/under at base (800, generally) ISO. Alexa has +7.3 or +7.8 depending on who you ask, Varicam +6.5, C300 Mk II +6.3, EVA1 +6, black magic does well, too, but I don't know if P6K should be rated at 400 ISO or 800 ISO base. Ursa 4.6K I think was around +5.8? F5 is +6 at 2000 ISO base, but I rate that camera at 800 ISO so it's more like +4.7 imo...

What's the over/under with this sensor?

Also, I downloaded the sample footage, can't play it back in either Resolve or braw player 1.6. I am on a Mac.

 

[AndreeOnline]

I've probably stated why here before but here goes again.

Alright, thank for the explanation.

I don't think anyone at least somewhat sane would rate a 5D mkII higher than a C300 mkII. I don't recall that article myself, but I'll take your word for it. If they had that test/result in their database it would invalidate the rest of them. But I don't find it and I'm only referring to the recent tests done in their "LAB" according to the same methodology. I think Gunther Machu has done them all (not 100% sure though). That old thorn in your side is most likely from a previous generations of tests?

Anyway, you are right about NR. It's a unneeded variable. A camera that punches above its weight in the DR department lately is the S1H (had to look up what brand it is: Panasonic). I understand it's partly because of similar reasons. The S1H actually IS in the database and does well there: 12.7 stops.

But I don't mind that at all, as long as I know the cameras are tested the same way. I'll make a mental note about NR and just file that test as one of many data points to use when reading about a camera/sensor. I also don't draw any other conclusions about the camera apart from noting their DR result (including some of the details, not just the final number). As you mention, there are many other things—including things that could influence the test result—that need to be looked at. And if the NR really is overly aggressive, it should show up in other tests where the camera might produce 'muddy' images or similar, i.e. pay the price in another department.

 

[Tom Roper]

Are you saying Arri is 15+? I heard from someone who spoke with an Arri rep that they'll admit this if you press them, but they don't want to muddy the waters.

More useful is over/under at base (800, generally) ISO. Alexa has +7.3 or +7.8 depending on who you ask, Varicam +6.5, C300 Mk II +6.3, EVA1 +6, black magic does well, too, but I don't know if P6K should be rated at 400 ISO or 800 ISO base. Ursa 4.6K I think was around +5.8? F5 is +6 at 2000 ISO base, but I rate that camera at 800 ISO so it's more like +4.7 imo...

What's the over/under with this sensor?

Those are just exposure indices for Log-C, BMD Film gamma, etc. It's not a performance metric, it's an exposure guide. Sensors are linear.

 

[Amulet Man]

Their number is actually at least a stop under what it really is. Even Arri lied

That's the point. There really isn't a standard way to measure DR. And the real elephant in the room is that most DR specs from a manufacturer are generally based on calculating from an idealised model of the sensor.

They do this because it means you don't have to test dozens of sensors to get a true baseline (news flash, every single sensor performs differently even on the high end) and then factor testing methodology and lenses etc.

And then the really true reason why you can't get standardisation. There's no way to quantitively measure noise. so some HUMAN BEING sits there and say, thats' too noisy, or that's acceptable noise.

And with three different noise patterns you generally get 4 different opinions of what's acceptable detail and what's not.

So they use maths. And that's the way they all do it. And that's why no-one ever can get close to what they claim. And why it's a kind of meaningless metric. Or least, one that's not that useful in the bigger picture and is more easily tested by use in the way you work.

And look at this. There were headlines screaming about RED doing 20 stops...this is a 2 generation old camera now, but here's the same chart showing 20 stops. Photographic proof it's all BS...
https://www.reduser.net/forum/showthread.php?92158-Enter-the-Dragon

So most don't sit there with a chart and calculate it, though Arri say they did with their number with a DR chart they designed but can only apparently document up to 15 stops and no-one else uses, but their number is off too...so ?


JB

Also, on the FF camera test you posted... what's lens C? I really like that one.

 

[Amulet Man]

Those are just exposure indices for Log-C, BMD Film gamma, etc. It's not a performance metric, it's an exposure guide. Sensors are linear.

I don't understand what you mean. Do you mean you can place middle gray arbitrarily so the dynamic range figure is meaningful but over/under is meaningless?

This may be the case in theory, but I've never found it was true in practice. Except for the F5, which I rate a bit slower and notice most people do, too, I've found 95% of content is shot at base ISO and that manufacturers choose base ISO to provide the best balance of image quality (subjectively, not as measured) and highlight detail.

This is Red's argument... rate the sensor at 3200 ISO and suddenly you have as much highlight detail as an Alexa. Maybe this is true with the newer Red cameras, I haven't used them much since Dragon, which was noisier than advertised.

 

[Eric Coughlin]

Anyway, you are right about NR. It's a unneeded variable. A camera that punches above its weight in the DR department lately is the S1H (had to look up what brand it is: Panasonic). I understand it's partly because of similar reasons. The S1H actually IS in the database and does well there: 12.7 stops.

I did a shoot where the producer brought his S1H to use as the b-camera to my C300 Mark II. I did not set the in-camera picture profile for the S1H and have not used it aside from on that shoot, but essentially both cameras were set to a rec 709 style profile (Wide DR on the C300) and judging by the LCD screen on each the C300 Mark II has considerably more detail in the highlights as shown in the LCD photos below. Again, the producer did not seem super camera savvy so he may have chosen a profile that limited the DR more than other rec 709 profiles would. Of course, there is a lot to be said for a camera's default picture profile or LUT setting looking good to avoid user error (such as Arri primarily just having one 709 LUT as standard which does look good).

 

[Amulet Man]

I did a shoot where the producer brought his S1H to use as the b-camera to my C300 Mark II. I did not set the in-camera picture profile for the S1H and have not used it aside from on that shoot, but essentially both cameras were set to a rec 709 style profile (Wide DR on the C300) and judging by the LCD screen on each the C300 Mark II has considerably more detail in the highlights as shown in the LCD photos below. Again, the producer did not seem super camera savvy so he may have chosen a profile that limited the DR more than other rec 709 profiles would. Of course, there is a lot to be said for a camera's default picture profile or LUT setting looking good to avoid user error (such as Arri primarily just having one 709 LUT as standard which does look good).

That's the profile you used. The S1H's Log C profile has around 1/3 stop less highlight detail than the C300 Mk II's Canon Log 2 and Canon Log 3 profiles, and cleaner shadows.

 

[Eric Coughlin]

That's the profile you used. The S1H's Log C profile has around 1/3 stop less highlight detail than the C300 Mk II's Canon Log 2 and Canon Log 3 profiles, and cleaner shadows.

For myself, I'm more interested in the capabilities of a camera's rec 709 profile since I shoot rec 709 on 65% of my shoots. Again, I don't know if that S1H was set to the best rec 709 profile it has available; I'm just saying I wasn't impressed with the level of highlight detail with whatever it was set to. Which isn't saying a whole lot due to the limited information, but perhaps someone more knowledgeable of the camera could chime in and clarify if there is a crappy rec 709 profile on that camera, in which case I'd still say shame on that camera for giving the users a crappy profile to mess up their footage with.

I recall the C100 had a Canon EOS profile that made it look like a 5D, in other words, crappy. Crunchy with crushed dynamic range. They probably did that because so many people still had 5Ds and were trying to match them up with the C100s at the time. At this point, there's not as much reason to have a profile that is that crushed.