PGA & ASC Camera Assessment Test

[Ryan Patrick O'Hara]

http://www.youtube.com/watch?v=HhHdRUs2Jn4

^ I can't embed... I'm an idiot.

http://findarticles.com/p/articles/mi_7152/is_200904/ai_n32325335/

Anyway, I was there tonight at the Academy's theater, and wow! Those cameras were fantastic! Any way you cut it... every digital camera was as good as film in some situation or in some characteristic. Maybe not overall as good as the film, but some situations, a few got close.

Seeing the test projected in one of the top theaters in the entire world was simply amazing. I think everyone who is interested in large scale productions and top shelf cameras ought to go if they do repeat screenings. I got to meet Micheal Goy, the president of the ASC and found out more about the BSC's camera assessment, and how other countries are doing the same thing.

The Red One was one of the cameras in the test, and it did better then I had thought. It wasn't at the top of the heap, that's for sure, but it had some really nice color reproduction in certains scenes, but then got gross in others. Was very peculiar and inconsistent as far as quality throughout the different tests. Might be because of its unique post characteristics. I'm not sure.

The test wasn't to see what was best, but just how the cameras handled different situations. Once you know how a camera works in certain environments, then you know which camera to choose and/or how to treat it or cater to it's needs. In my opinion, the camera which I liked the most and had repeatable and consistent comparable images to that of the 'control' of the tests (35mm film) was the Arri D-21. Which is amazing because that's not even the new Alexa d-21's! I had a nice conversation with Stephan Ukas-Bradley who is the Product Manager of Arri's Digital Production division, about the D-21 and the upcoming Alexa's... and the benefits of optical viewfinders. I can't wait for the new D-21's. Going to take these top digital cameras to a new level.

The F35 looked very nice most of the time, as did it's little bro, the F23. Viper had issues... poor guy. The Viper held highlights well, but during the main highlight test, the lens 'flare' (more like a flare/fog) was unbearably bad. To my knowledge lensflares are mainly the effect of the lens glass, and not the sensor (unless CCD streaks, or clipping, etc... which are just effects of highlights, not flares) so their choice of lens must have been a bad one. Most camera's had Zeiss Master Primes, but cameras like the Viper (which I think is 2/3") had different lenses (Digiprimes) and the Genesis of course had Primo's.

I'm tired. I'll come back tomorrow and clean this up into coherent findings. Feel free to ask questions about what I saw this evening, and I'll try answer them later.

-ryan

 

[Taylor Rudd]

I heard there were some issues at that test with the F35 having some type of internal light pollution. Of course I heard this from the Panavision boys who still have a grudge with the F35's likeness to the Genesis

I doubt any abnormal pollution made it to the cut, but how did the F35 compare to the Genesis?

 

[Ryan Patrick O'Hara]

That was never mentioned in the powerpoint, footage narration, nor the Q&A afterward. I'd say that was either rumors or a non-issue.

During the highlight test, there were noticable vertical streaking from three of the cameras, but nobody really said anything about it, so I figure that was commonplace? Was strangely obvious, and I'd have people in post remove the streaks. If I remember correctly it was found in the Genesis, F35, F23, which makes sense because they are practically family. A disfunctional family, but a family.

The Varicam had horrible streaking from the lightbulb, but they redid the scene later with a baffle behind the lens and the problem was remedied.

 

[Ryan Patrick O'Hara]

Here is the long winded take of what I remember:

The PGA and ASC conducted a side by side comparison, and 'round robin' comparison of 7 of the biggest digital cameras in the industry. The standard or control for the test was comparison to a Kodak 35mm film stock... I can't remember which stock(s) but I'll get back to you on that. My guess was 500t and 250d... just because that's pretty much standard.

Originally there were only 6 cameras chosen by the ASC, but the PGA insisted that the RedOne be included.

The seven cameras were:

1) Arri 435 w/Kodak 35mm film stock

2) Arri D-21

3) Panasonic Varicam (3700 I think. If not 2700)

4) Panavision Genesis

5) RedOne

6) Sony F23

7) Sony F35

8) Thompson Grass-Valley Viper

There were two scenes where all the cameras were shot side by side, on a monster eight camera rig or simply on adjacent tripods. This was great as the side by side split screen comparisons could have the EXACT same actions. The first scene was at the Jaws ride at Universal Studios. The location offered deep greens in the trees, red gas cans, morning light, yellow leaves, a blue billboard, and great texture/resolution in the distant tree foliage and water ripples. In the scene the water is moving (obviously), the mechanical shark is jumping around, and the gas can's are firing pyro effects into the air. Color reproduction, saturation, resolution, and latitude were key elements to look for.

I can't remember everything, but for this scene all cameras did a nice job. Exterior day shots allowed all lenses and cameras to have ample light so noise and sharpness were at their best levels (low noise, more sharp images).

Many of the digital cameras (can't remember which) happened to reproduce colors rather accurately, but almost always just a bit more saturated. The red gas cans and blue billboard always seemed a bit more punchy on the digital cameras except with I believe the Viper and not as much with the D-21?... As far as color reproduction, almost all were pretty on the nose. Some of the digital cameras had a bit more yellow in the foliage versus the film, but on a minor scale. Nothing that really infiltrated or distorted the other colors. The Red was actually disappointing in this test. It handled the highlights of the shooting flames the worst (almost all clipped) and reproduced the red gas cans as a bit orange... something that reappears much later in the tests.

The other side-by-side scene was at the desperate housewives set (I believe) a wide playground scene. The people are in relative shade, and the sun is hitting the pavement around. I don't remember anything HUGE from any camera in that scene.

There was a scene shot where the camra dollies with a man riding a bicycle parallell with a picket fence to show aliasing or strobing effects, but Micheal Goy said it was omitted from the presentation because none of the cameras really showed a great difference.

The rest of the scenes were shot in what they called 'round-robin' style. The scene was lit for a film camera given many different characteristics to help challenge the cameras and get results. Each camera had 1 cinematographer who travelled with the camera. The camera manufacturers were all invited to participate and many did. They even had say of who was the DP assigned to their camera.

Each camera also had a crew which followed it around to the other scenes. But... every scene also had a cinematographer who's job was to light the scene in the beginning and make sure the lighting conditions stayed consistent for ALL cameras.

It is important to note that three of the cameras are NOT full sensor. The Panasonic Varicam, Thompson Valley Viper, and Sony F23 are 2/3" chips, so the DoF was inherently greater. But the DoF really wasn't a major consideration during the test considering since the lighting had to stay the same for every camera, even the 35mm sized sensors had differeing DoF because of thier differing sensitivities. So if anything, the DoF differences spoke more of the cameras sensisivity versus the DoF.

Each setup had different goals and lighting challenges. I'll edit this post later and discuss the 'round-robin' setups.

 

[Stephen Mick]

That's all well and good, but did anyone get to see the footage from this other camera, circled in the picture?

 

[Marc Villafan]

That's all well and good, but did anyone get to see the footage from this other camera, circled in the picture?

Yes, I believe that cam pulled a sleeper and took the whole show. Dominating in all fields, particularly in color representation and deep depth of field.

 

[Nitsuj]

So are they still keeping the results to themselves on this one? Not sure why there is this shroud of secrecy around this assessment. Only thing I could think of was that there was a deal made with the camera companies to keep it out of public.

 

[Ryan Patrick O'Hara]

Secret? Nah, man.

The thing is... it's a subjective test. None of the cameras did much better then the others on all the tests/scenes. The results aren't simply, the Genesis and D21 were supreme and Red sucked or vice versa. You have to see for yourself and then look at the stats they did during the test like average setup time and etc.

Because I was at the screening, I can tell you that I liked the D-21 best, and they said the cheapest camera was actually a photochemical film release. Hahaha, which was cool. But with that said... there was one scene where I thought the Red did the best on color rendition... but then on another it didn't look nearly as good. Each camera performed in different ways in different situations.

Plus... they are doing more tests (jokingly called 1.5 or something) because some manufacturers (like Red and a few others) didn't like the post workflow in which all media was treated. Anyway, they are currently working on the Blueray DVD so that people can hopefully watch for themselves on a nice calibrated HD monitor and judge that way.

This is all being produced and paid for by the PGA/ASC... all the crew, equipment, film, etc was volunteered/ donated. This is taking some time because it's a small budget and people are working on this for cheap, just to contribute. Keep your ear to the ground for future showings, or the blueray release.

Sorry I never came back and edited the post... now I can't remember all the details, just major things.

 

[Patryk_Rebisz]

Ryan, on my next film i'm considering either Genesis (that i have used before) or F35 (that i have not). Based on this test do you have any insight to share about those 2?

 

[Nitsuj]

Ah makes sense thx

 

[Ryan Patrick O'Hara]

Ryan, on my next film i'm considering either Genesis (that i have used before) or F35 (that i have not). Based on this test do you have any insight to share about those 2?

They did really well. I was super impressed with the D21, Panny, and f35. From what I can remember the F35 and Genesis (and f23) shared lots of similar characteristics, no doubt from their similar (and not so distant) camera DNA. Nothing from memory really stood out as being hugely different. I kind of remember and don't hold me to this, the F35 had a tiny bit more color saturation in very high chroma areas whereas the Genesis did not reproduce heavy chroma as saturated. When I said tiny, i must stress that this was just noticable under utmost scrutiny. Probably less difference then variations from different viewing systems... ya know?

Either way, you're the luckiest bastard I know. The Blueray should be out by now since they said 'a couple of weeks' about two months ago. I'd call the ASC clubhouse and see what's good.

 

[Jon Lawrence]

More info on the PGA/ASC Camera Assessment

More info on the PGA/ASC Camera Assessment

Hey guys, just joined to chime in on some of Ryan's comments and to add to and clarify a few things:

1. As of two weeks ago, Panasonic has killed off the offer of making Blu-Ray's of the CAS. Unfortunately, with over a year of "all volunteer" effort, decisions didn't get made quickly enough for Panasonic to feel the test was still relevant. So as of this writing, there will be no blu-ray of the tests.
   
2. Because of the fact that the PGA and ASC were trying to please everyone (camera manufacturers) instead of just running an objective assessment and let the chips fall where they may, there were a lot of politics that are currently holding back general release. While I'm not directly responsible for that, I'm probably the only person who will offer an apology to all of you out there who would love to see the final results.
   
3. At this point, the rumblings on another CAS test are just that. Rumblings. I would be very surprised if we ever again got as much manufacturer participation as we did on the first shoot, and since it took us 3 years to get the first one done... well, don't hold your breath.

Lastly, I'll offer some of my own opinions. I was there in the color grading suite at Laser Pacific for much of the DI process. I saw raw footage, and what changes we could and couldn't make. So my quick notes are:

• Arri D21 was my hands down favorite - no artifacting or striping on the sensor when faced with difficult lighting challenge*
• The Panavision Genesis hands down won on workflow ease - from "mags" to on-set dailies review to 10-bit DPX files going straight to the post house. Easy as pie.
• However, the Genesis, the F35 and the F23 DID all have striping in the original footage from the light filament in one scene. It was less noticeable on the Genesis than the Sony cams, but not by much.
• The Arri 435b Film Camera, shooting on 500 ASA stock (I believe it was Kodak 5277?), even had some light artifacting (halo) around a single light source. Indeed, as it turns out, it seemed to be an actual reflection between the silver halide layer and the red emulsion layer on the film, creating a little warm, red halo around the light. *The D21 did have a slight, slight blue halo around the light, but not very noticeable.

I think the real takeaway is this - you CAN shoot a movie on any one of the cameras we tested. Some will move faster or slower in production (and hell, YOU can slow down production just as much if you don't know what the heck you want to shoot, but that's another discussion) and post than others. Personally, I'm a big fan of smaller, cheaper rigs than what we shot on, for the most part.

But I implore all of you, make sure your STORY is ready to be shot, then use whatever tools you have available. My friends and I successfully made, sold and delivered to market our last little indie feature on the trusty DVX100B. Nothing fancy, just a salable product.
How many times have you seen a movie or a TV show that looks great, but sucks? And how many times have you seen something that was GREAT show, but felt robbed because it didn't look great? Right.

Story, story, story, story! FIRST! Then tools to tell the story Rock on everyone.

 

[Ryan Patrick O'Hara]

Thanks Jon!

As always, you are right, the test was pretty political, but what do you expect with the PGA involved? hahaha! (jk)

I also doubt we will ever see a camera test done on this level again... as Jon mentioned, most manufacturers were simply unhappy with the way their cameras were tested. This isn't because the ASC/PGA tested them wrong necessarily... but each camera has an optimal workflow, and the ASC/PGA were trying to force all of them down the same relative standard pipe... something the RedOne and Viper really don't like doing, among the others.

I don't know... I think if we learned anything from this test... is that we need some kind of color and workflow uniformity throughout the production to post production world. The proprietary coloring systems which vary from post house to post house is something we should have solved long ago. I'm waiting for the ASC's CDL or something similar to really grip the post world.