Jay Nemeth
Member
I'm going to post a brief description of the testing process and a few comments while they are still sort of fresh in my mind. But first, some props are in order. Big thanks to Chuck Haifley at Big Vision Studios in Burbank (BigVision.com) for letting us use his entire facility including stage. He provided us with the F900, the Varicam, 14" Sony CineAlta CRT monitor, 23" Luma LCD monitor, Leader waveform/vector, HD-SDI router, and anything else we needed. Thanks to Engineer Damien, Michael May and everyone else there for their assistance.
Big thank you to Michael Kent with DSC Labs (dsclabs.com) for providing us with their best ChromaDumonde chart. Everyone should carry one of these with them so they can properly adjust their camera and shoot often to help editors and timers in post. Also check out their Tech Tips section.
More thanks to Michael Bravin at BandPro Film and Digital for loaning us their DSC Labs Ambi - Combi 2 illuminated resolution chart.
And thanks to Adam Wilt, the camera owners and ProMax guys for bringing all their stuff and expertise. Thank you actress/model Kacy Bult for being our "human" in the dynamic range torture test.
The 4 camera shoot out was really 6 camera. Since the buzz is about HD in an affordable camera, we need a point of reference, or actually 2 points - The Sony F900 and the Panasonic Varicam. In addition to shooting film, I use these 2 cameras quite often and do my own DIT work when I have to. I was more interested in how the HVX200 compared to the 2 big cameras while Barry was more focused on the "little" cameras. As it turned out, comparing 6 cameras was an enormous undertaking and we could have used a couple more days. There were so many things I wanted to do like the motion / compression torture test, but we just ran out of time.
Another aspect of the test that frustrated me, was that the cameras were so different in their design and operation, that it was sometimes impossible to adjust them to be the "same". They all had different detail circuits with ranges that were labeled differently, -7, 0, -9, off, and min were all terms that were supposed to describe the same thing - no artificial detail added. But in fact, some cameras couldn't completely turn their edge enhancement off (the Canon) and I am convinced the Sony Z1 goes beyond turning the detail off to actually introducing a high frequency filter at it's lower settings. There is no other way I can think of to explain it's loss of resolution at the lower settings. I've never seen anything like this before except on a DVD player.
Everybody has a different term for their gamma curves, Cine 1, Cine 2, Cine Like, Cine V, D, Film Out, etc. And these curves don't always do what you expect. When planning for a Film-Out, you typically stretch the blacks to get as much detail down there as possible while incorporating a knee circuit to keep the highlights from blowing out and restore some of the chroma. The JVC had a setting labeled Film Out that actually crushed the blacks, the exact opposite of what you would want. It's frightening that so many non technical people will be shooting projects with these cameras and make bad choices based on improper labeling of settings.
My goal was to get the cameras to match as much as possible, so we could analyze their performance in a given situation without miscalibration being an influence. Unfortunately, I do not know much about the settings, choices, labeling, and menu operation of the little cameras, so I relied on the owners to adjust their cameras and Barry to adjust my HVX. I set up the F900 and the Varicam and spent most of the day staring at the scope and monitors and consulting with Adam on how we should approach each test.
I'm going to leave the thorough report to Barry, especially since he has to review the playback. Important to note that during the day, we were only evaluating camera head / lens combination performance. We didn't look at playback to take into account various compression schemes. What I want to do is describe our testing procedure with a tiny bit of my observations.
All cameras are switched to their 24p mode with a 180 degree shutter or 1/48th of a second shutter ON. The exception is the Z1. Since it has no real 24 frame mode that doesn't significantly compromise the image, we decide to leave it in 60i mode.
The first test: How well do the cameras perform an auto-white. We momentary auto iris the cameras to find the place they want to be on the white card. Vectorscope set to X5 magnification. Good news, they all nailed the center dot on the scope. Unexpected result: Some of the little cameras exhibited a definite "digital" quality to their fuzzy white dot. Adam suspects it might be a chroma subsampling artifact. It looks to me to be some sort of aliasing.
Second test: Sensitivity. All cameras are aimed at 18% grey card, also called neutral grey. The card is illuminated with 120 fc of light which gives an f-stop reading on my Spectra Pro of 5.6 at 320 ISO, 24fps, 1/48th of a second exposure also called 180deg shutter. I choose this setting because it is what I rate the F900 at, and I'm really judging everything against that camera. The Varicam has many options that dramatically affect it's ISO rating, so I'm not comparing things to it from a subjective standpoint. The iris is adjusted so that the grey card reads 50 IRE on the waveform monitor.All cameras set to standard or normal gamma. Here are the results:
Camera . . . . F-Stop . . . . ISO rating
F900 . . . . . . 5.6- . . . . . . 320-
Varicam* . . . 4- . . . . . . . 160-
HVX200 . . . . 5.6 . . . . . . .320
Canon H1 . . 4 1/3 . . . . . 200
JVC 100 . . . . 5.6- . . . . . . 320-
Sony Z1 . . . . 4 . . . . . . . . 160
* Varicam is set for the best possible Film-Out mode and so has reduced sensitivity. It is usually in the 320 range.
The "-" symbol after a number means it was slightly under, but not by a full third of a stop.
We took readings from these cameras with various other gamma readings, but it is useless info since stretching and compressing the blacks usually drags middle gray with it. On the Varicam set to Film Rec with 500% Dynamic Level, the middle grey should read 30.5% IRE. Who knows what the little cameras should be set to. We could have goofed around with finding the best place for the 11 step greyscale to fit between 0 and 100 IRE, but this would have little effect on the sensitivity of the camera. The normal gammas show a linear response, and are the numbers we should be looking at.
Third test: Resolution using ChromaDumond reflective chart. This section requires a whole book to report what we saw, so I will let Barry give those results since he is so good at writing books. I will say that one of the objectives to get accurate results is to get all the cameras to a "detail off" setting. On the F900 and Varicam, that's easy because they have a menu item labeled......yes you guessed it, "detail off". The little cameras all have their own radically different ways of doing it and they're all all labeled differently. You would think the lowest setting would do it, but that wasn't the case. So, we ended up turning off the detail if the camera permitted, and recording data and footage to the G5 for review later. When the Canon wouldn't let us turn off detail completely, we raised the detail on the other cameras to match it so we could at least look at them set evenly. The Sony Z1 actually had to be turned up to get to a point that we felt represented a "no detail" condition. When it was set to minimum, it lost resolution, which is impossible if we were truly adjusting only detail. If anybody who designed or engineered that camera would like to explain what was going on, please chime in!
Fourth test: Ability to put Primary and Secondary colors in their appropriate boxes on the vectorscope. Using ChromaDumonde chart, all cameras passed (I think). Didn't take any notes on this part.
Fifth test: What exactly do the various gamma settings do to the gamma curve? Using ChromaDumonde 11 step greyscale we find that (and this all from memory):
The HVX Cinelike V acts as expected, and compresses the blacks a little to simulate a film-like image for HD display. The HVX Cinelike D stretches the blacks providing more detail and expanded exposure latitude for a possible Film-Out and/or more flexibility in post. We didn't test News Gamma.
The Canon H1 we have to ask Adam or Barry about because I don't remember. Sorry! Too busy staring at Waveform and not taking notes.
The JVC 100 had a setting called FilmOut that compressed the blacks. The exact opposite of what you would want. I don't remember if the other setting stretched. Maybe Nate can comment on this.
Sony Z1 compressed blacks on both settings, good for HD display if you like the contrast, but again, don't use these settings if you think you might make a print.
Sixth test: Resolution using Ambi Combi 2 back lit chart. Not a whole lot learned here beyond the other chart since none of the little cameras could even come close the 1000 lines maximum of the ChromaDumonde. Diagonal lines will be interesting to look at regarding pixel offset later on.
Big thank you to Michael Kent with DSC Labs (dsclabs.com) for providing us with their best ChromaDumonde chart. Everyone should carry one of these with them so they can properly adjust their camera and shoot often to help editors and timers in post. Also check out their Tech Tips section.
More thanks to Michael Bravin at BandPro Film and Digital for loaning us their DSC Labs Ambi - Combi 2 illuminated resolution chart.
And thanks to Adam Wilt, the camera owners and ProMax guys for bringing all their stuff and expertise. Thank you actress/model Kacy Bult for being our "human" in the dynamic range torture test.
The 4 camera shoot out was really 6 camera. Since the buzz is about HD in an affordable camera, we need a point of reference, or actually 2 points - The Sony F900 and the Panasonic Varicam. In addition to shooting film, I use these 2 cameras quite often and do my own DIT work when I have to. I was more interested in how the HVX200 compared to the 2 big cameras while Barry was more focused on the "little" cameras. As it turned out, comparing 6 cameras was an enormous undertaking and we could have used a couple more days. There were so many things I wanted to do like the motion / compression torture test, but we just ran out of time.
Another aspect of the test that frustrated me, was that the cameras were so different in their design and operation, that it was sometimes impossible to adjust them to be the "same". They all had different detail circuits with ranges that were labeled differently, -7, 0, -9, off, and min were all terms that were supposed to describe the same thing - no artificial detail added. But in fact, some cameras couldn't completely turn their edge enhancement off (the Canon) and I am convinced the Sony Z1 goes beyond turning the detail off to actually introducing a high frequency filter at it's lower settings. There is no other way I can think of to explain it's loss of resolution at the lower settings. I've never seen anything like this before except on a DVD player.
Everybody has a different term for their gamma curves, Cine 1, Cine 2, Cine Like, Cine V, D, Film Out, etc. And these curves don't always do what you expect. When planning for a Film-Out, you typically stretch the blacks to get as much detail down there as possible while incorporating a knee circuit to keep the highlights from blowing out and restore some of the chroma. The JVC had a setting labeled Film Out that actually crushed the blacks, the exact opposite of what you would want. It's frightening that so many non technical people will be shooting projects with these cameras and make bad choices based on improper labeling of settings.
My goal was to get the cameras to match as much as possible, so we could analyze their performance in a given situation without miscalibration being an influence. Unfortunately, I do not know much about the settings, choices, labeling, and menu operation of the little cameras, so I relied on the owners to adjust their cameras and Barry to adjust my HVX. I set up the F900 and the Varicam and spent most of the day staring at the scope and monitors and consulting with Adam on how we should approach each test.
I'm going to leave the thorough report to Barry, especially since he has to review the playback. Important to note that during the day, we were only evaluating camera head / lens combination performance. We didn't look at playback to take into account various compression schemes. What I want to do is describe our testing procedure with a tiny bit of my observations.
All cameras are switched to their 24p mode with a 180 degree shutter or 1/48th of a second shutter ON. The exception is the Z1. Since it has no real 24 frame mode that doesn't significantly compromise the image, we decide to leave it in 60i mode.
The first test: How well do the cameras perform an auto-white. We momentary auto iris the cameras to find the place they want to be on the white card. Vectorscope set to X5 magnification. Good news, they all nailed the center dot on the scope. Unexpected result: Some of the little cameras exhibited a definite "digital" quality to their fuzzy white dot. Adam suspects it might be a chroma subsampling artifact. It looks to me to be some sort of aliasing.
Second test: Sensitivity. All cameras are aimed at 18% grey card, also called neutral grey. The card is illuminated with 120 fc of light which gives an f-stop reading on my Spectra Pro of 5.6 at 320 ISO, 24fps, 1/48th of a second exposure also called 180deg shutter. I choose this setting because it is what I rate the F900 at, and I'm really judging everything against that camera. The Varicam has many options that dramatically affect it's ISO rating, so I'm not comparing things to it from a subjective standpoint. The iris is adjusted so that the grey card reads 50 IRE on the waveform monitor.All cameras set to standard or normal gamma. Here are the results:
Camera . . . . F-Stop . . . . ISO rating
F900 . . . . . . 5.6- . . . . . . 320-
Varicam* . . . 4- . . . . . . . 160-
HVX200 . . . . 5.6 . . . . . . .320
Canon H1 . . 4 1/3 . . . . . 200
JVC 100 . . . . 5.6- . . . . . . 320-
Sony Z1 . . . . 4 . . . . . . . . 160
* Varicam is set for the best possible Film-Out mode and so has reduced sensitivity. It is usually in the 320 range.
The "-" symbol after a number means it was slightly under, but not by a full third of a stop.
We took readings from these cameras with various other gamma readings, but it is useless info since stretching and compressing the blacks usually drags middle gray with it. On the Varicam set to Film Rec with 500% Dynamic Level, the middle grey should read 30.5% IRE. Who knows what the little cameras should be set to. We could have goofed around with finding the best place for the 11 step greyscale to fit between 0 and 100 IRE, but this would have little effect on the sensitivity of the camera. The normal gammas show a linear response, and are the numbers we should be looking at.
Third test: Resolution using ChromaDumond reflective chart. This section requires a whole book to report what we saw, so I will let Barry give those results since he is so good at writing books. I will say that one of the objectives to get accurate results is to get all the cameras to a "detail off" setting. On the F900 and Varicam, that's easy because they have a menu item labeled......yes you guessed it, "detail off". The little cameras all have their own radically different ways of doing it and they're all all labeled differently. You would think the lowest setting would do it, but that wasn't the case. So, we ended up turning off the detail if the camera permitted, and recording data and footage to the G5 for review later. When the Canon wouldn't let us turn off detail completely, we raised the detail on the other cameras to match it so we could at least look at them set evenly. The Sony Z1 actually had to be turned up to get to a point that we felt represented a "no detail" condition. When it was set to minimum, it lost resolution, which is impossible if we were truly adjusting only detail. If anybody who designed or engineered that camera would like to explain what was going on, please chime in!
Fourth test: Ability to put Primary and Secondary colors in their appropriate boxes on the vectorscope. Using ChromaDumonde chart, all cameras passed (I think). Didn't take any notes on this part.
Fifth test: What exactly do the various gamma settings do to the gamma curve? Using ChromaDumonde 11 step greyscale we find that (and this all from memory):
The HVX Cinelike V acts as expected, and compresses the blacks a little to simulate a film-like image for HD display. The HVX Cinelike D stretches the blacks providing more detail and expanded exposure latitude for a possible Film-Out and/or more flexibility in post. We didn't test News Gamma.
The Canon H1 we have to ask Adam or Barry about because I don't remember. Sorry! Too busy staring at Waveform and not taking notes.
The JVC 100 had a setting called FilmOut that compressed the blacks. The exact opposite of what you would want. I don't remember if the other setting stretched. Maybe Nate can comment on this.
Sony Z1 compressed blacks on both settings, good for HD display if you like the contrast, but again, don't use these settings if you think you might make a print.
Sixth test: Resolution using Ambi Combi 2 back lit chart. Not a whole lot learned here beyond the other chart since none of the little cameras could even come close the 1000 lines maximum of the ChromaDumonde. Diagonal lines will be interesting to look at regarding pixel offset later on.
