The single chip camera evaluation by ASC members

B

Barry_Green

Moderator
The long awaited Single Chip Camera Evaluation test results were screened tonight in front of a crowd of over 400 people. This test is the one that Zacuto is doing a documentary of. It was headed up by Robert Primes, ASC and was a comparison of a dozen large-sensor single-chip cameras, ranging from the Arri Alexa and Sony F35 And 35mm film, to the Sony F3 and Panasonic AF100 to the nikon D7000 and Canon 5d, 1d, and 7d.

Really excellent tests. Over a hundred people, including several ASC members, worked to thoroughly evaluate these cameras and put them through rigorous tests And had them shoot challenging real world scenes to evaluate their performance. Each manufacturer was invited to send an expert to administer their cameras and ASC cinematographers supervised it all. They used the same lens on every camera (lenses proven to resolve more than 4K of resolution).

They didn't state subjective conclusions. They posted the results of their objective measurements (such as dynamic range, etc) and they posted side by side examples from each test to let the viewer draw their own conclusions. They also provided some background info on the purpose of each test and they gave some guidance on what to be looking for. And, it included perhaps the most superb visuL explanation of rolling shutter artifacts I've ever seen.

Overall the test was extremely well done. They will be making it availAble at thescce.org on blu-ray. If you are interested in separating the hype and rumor and buzz away from the reality and want to see how these products REALLY perform, in the hands of skilled professionals, in an utterly unbiased, thorough and complete evaluation, then I highly recommend it.
 
I bet this was a lot of fun to attend. It's a shame that the Red/Epic wasn't included, or maybe it was? Anyway looking forward to see the results.
 
From http://provideocoalition.com/index.php/shullfish/story/single_chip_camera_evaluation/:
film actually has phenomenal highlight detail, but isn’t actually that good in the shadow detail
Click to expand...

If you exposed video so that it had as much noise in the picture as film does, how much more highlight detail would you have? All video cameras have very clean pictures, cleaner than film, but they have much less highlight headroom. For example, I would just guess that most video cameras put the mids (faces and such) at just three or four F-stops below pure white. But to have enough room to apply a curve at the top, to roll off the highlights, you need at least a couple more. Which basically means you expose a couple stops darker overall while shooting, then bring things up in post, curving the top and bottom (the famous S curve). You really need to record at 10-bit or higher though, because banding can happen with just an 8-bit picture if you stretch it out.
 
Spartacus said:
So Barry, what´s YOUR conclusion? :cheesy:
Click to expand...
My reaction was that it was pretty reassuring to have a group of top industry professionals validating everything I've been saying for the last eight years. :thumbsup:

Seriously, there were a few surprises and mostly it came out exactly as one would (or should) expect. If you compare them by pricetag and year of release, you could fairly rank them. But not always. The f35, for example, didn't fare nearly as well as I expected, the f3 pretty much handed it it's butt on a silver platter. The red mx did pretty well, definitely besting the f35 and most of the others, and dominated in resolution. The Alexa stole the show. It was just astounding. It was dominant and to my eyes the clear winner, and everyone I spoke to agreed. Just no comparison. And film? Well, that was one of the disappointments. Film just didn't dominate. Always good, but (except for res) the Alexa was always as good or better.

For a budget shooter the f3 and af100 did extremely well. The f3 was indeed a baby Alexa, and the af100 was a baby f3. The f3 and af100 were always shown back to back and on most tests the af100 definitely held it's own. Not as good, for sure, but at 1/3 the price it acquitted itself very well. Only place it choked was on the 7-stop-overexposed window test.

The DSLR group was consistently at the bottom of the pack and people actually broke into laughter when the DSLR shots came up. All four looked quite similar, but the 1d had the best rolling shutter by far of that group. Several folks thought the nikon looked better than the canons.

One question was definitively settled: the assertion that "film skews like a rolling shutter" was proven to be utter horsecrap. On the rolling shutter tests film looked extremely solid and just like a global shutter. No noticeable skew of any type. Alexa was extremely minimal, leagues better than the red/f3/af100/1d, were all way superior to the three DSLR cams.

As for color, all the cams looked pretty similar and realistic except for the canons which had their very different palette.

Watching these on the big screen, I would absolutely be confident shooting a theatrically released feature on any of them except the DSLRs. There was such a gap between the video cams and the slrs in sharpness and clarity that it was very noticeable and caused the audience to laugh. And the gap in rolling shutter was significant too. But if I could buy any of them, hands down and no question: the Alexa. Best bang for the buck - af100. If you can afford the f3 is great and was very close to the top tier of cameras tested. Sony should be quite proud of it, although the f35 wasnt nearly as impressive considering the huge price gap. If your budget can afford an f3, it is definitely the bang for the buck winner for you. If you can't afford the f3, the af100 was pretty close and was definitely the overall bang for the buck winner.

And yeah, I have five movie cameras now. Four will be up for sale when I get home. I love love love film, but the writing is on the wall and the torch has been passed.
 
And yeah, I have five movie cameras now. Four will be up for sale when I get home. I love love love film, but the writing is on the wall and the torch has been passed.
Click to expand...

I wonder if it's demise will come about quite quickly. I've been skeptical until recently, that we'll see it's demise in the next couple of years, but I do think it's possible.
 
It's too bad the FS100 wasn't available either, I've been really curious as to how the image quality performs between the F3 and the AF100. And SERIOUSLY thank you for doing the rolling shutter film test. My contact for doing that hasn't been able to free up the resources for me to do that test, but if you are satisfied with what you saw, then I'll take your word on it. I think I want to get the Bluray though, as I'd really like to see these differences. I'd like to share with the local film people so they can actually see the quality difference people like me are trying to explain.
 
Barry_Green said:
The DSLR group was consistently at the bottom of the pack and people actually broke into laughter when the DSLR shots came up.
Click to expand...


Vincent Laforet sat through the exact same documentary and said on Teradek.tv ...

"I'm still not a fan, unfortunately, of the AF-100 at all and these tests confirmed a lot of what I've seen but in a more scientific way...

"The Canons held their own..."

("QA with Vincent Laforet - Day 2" -- http://teradek.tv/)

Everyone seems to agree the Alexa and the F3 were impressive.

The point here is NOT that Mr. Green is wrong or Mr. LaForet is right -- rather -- that there's room for differing conclusions.

Regarding Canon DSLRs on the big screen -- it's already been demonstrated they can work well on a large screen -- content by multiple artists displayed on multiple projection formats has been consistently well received.
 
Barry_Green said:
And yeah, I have five movie cameras now. Four will be up for sale when I get home. I love love love film, but the writing is on the wall and the torch has been passed.
Click to expand...

That's interesting Barry. Would you mind sharing which of the 5 you're keeping as a "movie camera" and why? Thanks.
 
Alexa has really been winning people over since it's release. I'm still an MX shooter but can't wait to play with Alexa a little more. Thanks for the report, Barry!
 
Individual opinions will vary. The test purposely avoided making conclusions or declaring a "winner". They leave that up to you. Get the blu ray and evaluate it for yourself. I know what I saw and how I felt about it. The af100 fell down in the chroma clipping but held it's own everywhere else. The canons fell down on the sharpness test but held up well in dynamic range.

People have a tendency to see what they want and that will happen here too, which is why they leave the evaluation up to the individual. I am quite comfortable with what I posted and I have talked to many who were there.

If anyone is confused about the results I highly recommend getting the blu ray and viewing the actual results and forming your own opinion.
 
You must log in or register to reply here.
Back
Top