The AF100 Great Rolling Shutter mystery....

Rick Burnett

Rick Burnett

Section Moderator
For those of you that saw the BBC Alan Robert's test report of the AF101, one of the things I just didn't understand was why his read/reset times were not matching mine at all. I was very careful in my calculations.

So, before I remeasured my camera, I downloaded the PDF and extracted the two frames Alan used to generate his measurement of the read/reset. Given I was using a scaled version of his original files, and that I had to extrapolate the line of the window to the bottom of the frame, there is a little room for error. I also noticed that the slope of his windows does not match in both frames, which will account for additional error due to the pan speed not being constant.

All that said, I measured 30.3ms of read/reset. Given all I mentioned above, I believe what he reported on the camera he was given must have been correct. He was using, from what Adam Wilt has told me, was a preproduction AF101.

Given that I pretty much validated his images, I decided to redo my test in both 50Hz/25p and 60Hz/24p. I tried to be very meticulous on this test. Here are the results:

50p ~17.25ms
60p ~14.85ms

Given this is the second time I have measured the 60p and both are in the 14ms range, I feel even more confident on my findings. By the way, to learn more on this method of determining the read/reset, check here:

http://provideocoalition.com/index.php/awilt/story/overview_several_first-generation_versacams/P1/

Also, in the comments, Adam has also tested the Red One.

Could this be a difference in the AF100 and AF101 models? Or, can someone confirm that Alan did his tests with a pre-production model? If so, this disturbs me because people aren't getting the full story on what the AF100 is capable of. I know for sure that the noise signatures I saw on the preproduction were MUCH MUCH worse than what I see on the production model. In fact, I have never seen that noise since (and I verified this noise on Jan's camera playing back footage to a Panasonic broadcast monitor, so I know it existed in the preproduction model).

I honestly believe that Alan performed an unbiased and straightforward test of the camera. That said, he mentions that the tests he performed were on two cameras with no serial numbers.

Anyways, I think this is important to bring up.
 
I just wanted to add something interesting as well. In looking at these numbers, it looks like the read/reset is tied to the operating frequency of the camera, which makes sense of course. Given the operating speed of 50Hz is slower, it takes longer for the read/reset. They didn't have to do it this way, but that would be my guess looking at the values I got. Makes me glad I shoot in 24p!
 
Interesting. Shouldn't we all shoot in 24P mode then (unless we are required to deliver to a broadcaster in PAL-land)? I live in a PAL country, but happily shoot 23.98 a lot of the time for my work.
 
Rick Burnett said:
All that said, I measured 30.3ms of read/reset. Given all I mentioned above, I believe what he reported on the camera he was given must have been correct. He was using, from what Adam Wilt has told me, was a preproduction AF101.
Given that I pretty much validated his images, I decided to redo my test in both 50Hz/25p and 60Hz/24p. I tried to be very meticulous on this test. Here are the results:
50p ~17.25ms
60p ~14.85ms
Click to expand...

Hi Rick, I don't understand which of the two measurements you are saying is correct. Is it 30.3 or 14.85?

Also if Alan had two cameras and neither had a serial number, then they would be prototypes, the factory does not ship production level cameras without serial numbers. That said though I have 15 cameras that do not have serial numbers and they currently are running the production level firmware.

You also said:

(and I verified this noise on Jan's camera playing back footage to a Panasonic broadcast monitor, so I know it existed in the preproduction model).
Click to expand...

I am assuming by Jan you are meaning me. Which footage would that be and where did you see it?

Thanks,

Jan
 
Hey Jan,

On my specific camera, I have measured in the ~14.85ms range multiple times in 24p and am confident that this is correct. This was the first time I measured 25p as I do not normally ever have my camera in 50Hz mode. I was VERY careful in my testing and even with the small margin of error I believe there is in this test for pan speed differences, this does not account for the difference that me and Alan Roberts is seeing. I have asked Alan if it was possible to get his footage, but I know that he is busy and given the time that has elapsed, he may not have it either.

I took the two images from his report, and used the same method to determine from the subsequent frames to determine what the skew was in his shot, and I can say that the value I get is consistent with his calculation, which is what surprised me. His two images show a significant amount more of skew (almost double).

Pertaining to the noise in camera, the Panasonic representative that was at a lighting trade show in Winsten-Salem, NC told me the specific AF100 he had belonged to you. On that AF100 was the demo footage containing many of the video segments that were part of the noise complaints from the pre-production shots that some had posted. This included the girl lit by candlelight and the girl in the black skirt, among other shots. They also had a Panasonic monitor hooked up to the camera. I was able to not only view these pieces of footage on your camera and on the Panasonic monitor, I was also able to observe your camera shooting at the event. In both of these instances, I saw more noise in the footage even after adjusting settings on your camera that I do not see on mine. This noise was consistent with what I saw on playback of material I had downloaded or viewed after some reviewers had posted it. All that said, I've tried to push my camera to get that same noise I saw, and I have NEVER even with the default settings seen that much noise in my AF100.

I am honestly baffled.
 
sean90291 said:
Interesting. Shouldn't we all shoot in 24P mode then (unless we are required to deliver to a broadcaster in PAL-land)? I live in a PAL country, but happily shoot 23.98 a lot of the time for my work.
Click to expand...

That would be perfectly fine if you wanted to shoot everything in daylight but the moment you go indoors, and shoot under conventional mains powered lighting, you'll get rolling black bars in your footage (due to the camera's base frequency being 60Hz).
 
Recently, at NAB in Las Vegas, there was a presentation called SCCE single chip camera evaluation that was Hosted by Robert Primes ASC and others. Cameras evaluated were Alexa, Red MX, F35, Phantom ,Weisscam, F3, AF100, Canon 5D, 7D, 1D Mk 4, D7000 and of course 35 mm film.

Included in the extensive tests done to each of theses cameras was a visual rolling shutter test with a spinning drum and a spinning disk. I found this test to be very revealing , much more so than calculated numerical value. At the end of the presentation Mr Primes said that the data from the test will made public and that the whole presentation film will be made available on blu ray in the future. I would keep an eye out for the presentation when it becomes available.
As far as rolling shutter is concerned, the AF was among the better cameras in the test. Some cameras did not have any apparent rolling shutter , then there was a group that had some, and followed by the DSLRS which had the worst rolling shutter.
 
Well given I finally had four hours of sleep, I went back and looked at the original images that Alan Roberts posted in his report and decided to focus on the horizontal lines. One thing he has not provided is what a zero movement frame looks like with regards to the windows. What I can see is the camera is not such that the bottom of the sills is perfectly perpendicular with the horizontal plane of the sensor. In fact, even looking at the calculations that he gets more skew, it is consistent with the direction of the leaning of the window sills on the horizontal plane. There are unknowns with his test that I just cannot verify.

In all my tests, I was VERY VERY careful to make sure that when there was not panning happening, the line of question was exactly vertical at both the left and right sides of the frame as it exited. If I had to error a little, I made sure the left edge was exactly perfect and the right edge within a few pixels top to bottom. I also tested closer to the left edge in the frames I chose.

As to some background of why my interest in rolling shutter values has been renewed, I have heard from a good number of people who actually believe the rolling shutter on the AF100 has not improved over the GH1/GH2. They quoted the read/reset from the Alan Roberts report and given his level of experience/acceptance, refuse to believe any different. I've used a GH2 and compared it to the AF100, so I know personally the AF100 is better. I also tested the AF100 both through measurement and actual usage on a few shoots now. For me, the difference is night and day. I could NEVER do handheld work on my 7D with an 85mm lens, and yet, on the AF100 it looks great. And this is with the slightly tighter crop on the AF100 which would make it even worse than the 7D.

One of two things have to be going on.

1) The cameras Alan Roberts used had something going on with them, maybe firmware, maybe hardware.

2) A mistake was made in the camera setup. Maybe the camera got bumped and the vertical lines were not correct during this measurement.

I've done the test twice now for 24p and both times the values came out within the ~14 to ~15 range. Both shots were at wildly different distances from the subject, and I used two different exposure times (because sharper is better) of 1/500 last night and 1/2000 for the original test I believe. I've posted the method as well that I derived from Adam Wilt in case there is just something he and I are missing. I've double checked his methodology and believe it to be accurate.

This also goes with the noise issues in the past I posted, maybe it was preproduction firmware, maybe it was a hardware thing, but I don't see it on my camera. I find it hard to believe I have a special camera, but I am certainly glad that my camera performs so well. It's not often I will go to this level to get to the bottom of something, but I think Panasonic is getting a bad rap for something I don't believe to be true. I think their rolling shutter is actually VERY VERY good and it should be used as a selling point for those who feel limited by the rolling shutter of their DSLR. Period.

Mark, I plan on getting the BluRay once they release it! I am really interested in seeing it!
 
Rick Burnett said:
On my specific camera, I have measured in the ~14.85ms range multiple times in 24p and am confident that this is correct.
Click to expand...

I would concur.

This included the girl lit by candlelight and the girl in the black skirt, among other shots. They also had a Panasonic monitor hooked up to the camera. I was able to not only view these pieces of footage on your camera and on the Panasonic monitor, I was also able to observe your camera shooting at the event.
Click to expand...

Even more interesting, that same footage we used at NAB, and no one complained about it having noise. It was used in the theater, big screen, at least the green screen section and assortment of trinkets and it just looked terrific. Sometimes LCDs can be noisier, and depending on which one that could account for some variation. We were using plasmas, which show noise in a different way as well, and the projector, I don't recall the technology on that one, it may have been a DLP. Regardless it looked pretty good.

Not sure you are going to find reason for the differences.

Best,

Jan
 
All I can hope is that someone else makes a measurement at some point and there becomes at least another data point to compare. I guess the only other thing I could do is mount my AF100 below my 7D and align the image planes, then use a zoom on the 7D and match the framing to the line of interest exactly. Then pan both and provide those results to people as images. It wouldn't prove what the rolling shutter is specifically in read/reset ms time, but it would clearly show a measurable delta between the 7D and the AF100 in terms of pixels. Maybe I will do that when I have a long AE render writing out :)

And about the noise, like I said, I am baffled as well. I agree that the LCD on the AF100 can look noisier that the footage. This happened to me at a recent shoot (and surprised me as well) as the footage on playback 1:1 I couldn't see ANY noise in the same places. Which of course, if I were to err, it would be in this way! I was able to view that footage you projected at NAB on two different Panasonic LCD monitors at the lighting show that definitely were designed for accuracy it appeared. What I don't know is how well they were calibrated, but I believe I asked about one of them and was told it was (It looked like a 7 or 9 inch panel). On both I saw dancing noise in the blacks. I even had Barry send me footage from a pre-production AF100 and also saw some of that same dancing noise in the grays. Yet, with my production camera, which is newer firmware, I don't get that same noise in the grays. You're probably right though, I may never know :) Given projectors seem to have smaller contrast ratios in general, maybe that's why it never shows up when projected? Just making a guess. Given how much time we focused on the noise issue that doesn't exist anymore, I really didn't want to rehash that. My only reason for even including it here was I feel there was a difference between the preproduction and production cameras, and this made me suspect to the rolling shutter as well if Alan had used preproduction cameras in his tests.
 
What I'd really like to find is someone with a GH2 and someone with a Red One MX that will generate the test data for me and let me measure it, just because I don't have the GH2 metrics and I want to compare the Red One MX metrics to what Adam calculated.

The test is pretty easy to set up. First, you want your shutter as fast as possible and I tend to use lenses around 85mm that generate a skew easier. You set your camera up to view a line on something that has a high contrast edge vertically that extends through the whole frame, and you make sure that if you slowly pan your camera to the left edge, the line is exactly parallel to the left edge of the frame, and make sure the right edge as well. It can be as simple as a printout of a line. The higher the contrast, the better, and it as to be sharp in focus.

Then, you just pan the head back and forth across the line trying to get a smooth pan that does not accelerate or decelerate through the pan across the line. That's it.

If someone wants to post some footage from either camera (GH2 or RedOneMX) I will do the calculations (or you can) :)

(And for the GH2, I'd like normal 24p and the tele mode 24p to see if the skew is different between them)
 
Last edited:
I ran the gh2 and the af100 both through The Skewer and can definitively say that there is a huge difference in rolling shutter performance between them when shooting 1080/24p.

I also of course viewed the SCCE test and can confirm that there were pretty much three categories:
1) no skew. All the ccd cameras, Alexa, and 35mm film fit here.
2) medium skew: the red, f3, af100 and 1d were in this group and all seemed about equal.
3) most skew: the 5d, 7d and nikon made up this group. I would expect the gh2 to fit in this category.
 
I may be able to get a hold of a RED MX later this week and try it out. I know my friend is shooting with his through Wednesday, but I may be able to grab it for an hour or so late in week and give it a whirl, and hopefully attach the AF100 to it as an established baseline. No promises though, as I know we both have a busy week and am not sure I will be able to do it. Will see what I can do though.
 
Rick, I sent you a PM.

I tested the AF100, GH2, and GH2/ETC mode in 1080/24p. On the Skewer, the AF100 delivered 91 pixels worth of skew, and the GH2 was 153 in the same circumstance. The GH2 in ETC mode was 85.
 
Barry_Green said:
Rick, I sent you a PM.

I tested the AF100, GH2, and GH2/ETC mode in 1080/24p. On the Skewer, the AF100 delivered 91 pixels worth of skew, and the GH2 was 153 in the same circumstance. The GH2 in ETC mode was 85.
Click to expand...

Calculating skews now based off of your numbers.... :)
 
Oliver Rush said:
I may be able to get a hold of a RED MX later this week and try it out. I know my friend is shooting with his through Wednesday, but I may be able to grab it for an hour or so late in week and give it a whirl, and hopefully attach the AF100 to it as an established baseline. No promises though, as I know we both have a busy week and am not sure I will be able to do it. Will see what I can do though.
Click to expand...

I determined that the person I was questioning about the Red One MX performance was not doing full resolution (like 4.5k or 4k) and instead was looking at 3k. Given this, as you go smaller on the Red One MX (and probably the Red One) skew gets MUCH faster. Shooting at 2k is like 7ms!!
 
Rick Burnett said:
I determined that the person I was questioning about the Red One MX performance was not doing full resolution (like 4.5k or 4k) and instead was looking at 3k. Given this, as you go smaller on the Red One MX (and probably the Red One) skew gets MUCH faster. Shooting at 2k is like 7ms!!
Click to expand...

Well, I will try it with five different resolution settings and let you see. Makes sense that it would skew less at 2K though, it crops the frame at 2K so a much smaller area to deal with not to mention less work on the processor.... I wonder if the skew changes with the different redcode settings as well? Anyway, I will give it a shot at 4.5K, 4k 16:9, 4K 2:1, 2K 16:9, and 2K 2:1 in highest and lowest settings if I get a chance to play with it this week.
 
In asking Red about this, the only thing that affects the read/reset times is the number of horizontal lines you are reading. The less lines you read, the faster it can read all of them. This is not true on all cameras, as they don't all employ the same methods. From the measurements I have seen, I believe what Red has said. :)
 
You must log in or register to reply here.
Back
Top