Measuring rolling shutter: put a number on this issue!

Samuel H

Samuel H

Major Contributor
So, in the D5200 thread here, we're kind of worried that this exceptional little toy may have more rolling shutter issues than other cameras.

This should be a measurable thing: if we knew how many ms it takes each camera to read the sensor, we wouldn't have to be looking at all those piles of not-really-comparable youtube tests.
Also, when a new camera comes out, we would just have to measure it: no need to compare the D5200 first against a GH2, then against a D800, then a 5D3, then...

In order to get a half-decent measure of this, I would need to get a very short clip from each camera, shot following these instructions:
* camera on a tripod, well levelled, set at 24p and 1/50s shutter
* a vertical line somewhere in the frame, crossing it completely from top to bottom (it can be a pole, or a corner, or a door, etc; if it is clearly tilted it will distort my measure)
* start recording, do a quick pan left at constant speed, then a quick pan right at constant speed, repeat, reapeat, then stop recording
(that's a total of 3 pans left and 3 pans right, which will help average off the measurement errors)
(to get a constant speed, do a long pan that goes on for a while, even if it has to be slower, instead of just a quick whip-and-stop; if you can't find two consecutive frames where the skew looks the same, then the pan is not good)

If you want to know how your camera fares in relation to others, record that, then send me a link to the unedited file to similaar.feedback@gmail.com (or through a PM on dvxuser) and I'll use two consecutive frames to measure top-to-bottom delay in relation to frame-to-frame delay.

EDIT, because lots of people write to ask how I measure this:

First I calculate the horizontal frame-to-frame displacement, e.g. at the top of the frame. Say that's 90 pixels. That means the same portion of the image is moving 90 pixels every 42 ms (actually, 1000/24).
Then I go to the first of those two frames and measure the horizontal displacement within the frame, from top to bottom of that vertical line. Say that's 60 pixels, which, at 90 pixels every 42ms, means a first rolling-shutter measure of 28 ms.
Same with the second of those two frames. Say that's 70 pixels; our second RS measurement is 32ms. This means that the camera is accelerating a bit, that's why it's important to measure both frames: there's an error, but it will be averaged out (this is a first-order approximation: if the acceleration itself is accelerating, then that second-order error won't be averaged out, but in most cases that's going to be a very small factor).
Then look for another pair of frames where the image is moving in the opposite direction, and repeat those three measurements.
Say our measurements are: 105 for the inter-frame displacement, and 65 and 75 for the intra-frame ones, which gives us measures of 26 and 30 ms. So, the camera was faster this time but we don't really care because we're translating everything into time units; it was accelerating again this time, but as before that's not a (big) problem; and we also know that the vertical line was not perfectly vertical, but that's why we take measures in both directions, again this errors will be averaged out.
So, the average of those four numbers is 29ms.
I repeat everything again two or thee times more, because manually measuring blurry images is not an exact science, and that's how I get the final number that I post here.





BM UMP2 2k ------------ 3.2 ms (official)
RX100 V 250 fps ------- 3.8 ms (3.8-3.8)
BM UMP 12k - 4k crop -- 3.9 ms (official)
C700 FF s16 crop ------ 5.2 ms (provideocoalition)
a7R III 1080 S35 ------ 6.1 ms (6.6-5.7)
BM UMP 12k - 8k 2.4 --- 6.1 ms (official)
BM 4.6K (2K crop)------ 6.3 ms (official)
BM UMP2 4k ------------ 6.3 ms (official)
1DX II 1080p 60&120 --- 6.7 ms (6.6-6.7)
XT3 1080p 120fps ------ 7.4 ms (7.4-7.5)
BM UMP2 4.6k ---------- 7.6 ms (official)
a7R III 1080 FF ------- 7.7 ms (8.7-6.8)
BM p4k HD crop -------- 7.8 ms (official)
BM UMP 12k - 6k crop -- 7.8 ms (official)
BM UMP 12k - 8k/4k ---- 7.8 ms (official)
NX1 1080p ------------- 7.9 ms (7.7-8.0-8.1-7.8)
BM p6k 2.7k crop ------ 8.0 ms (official)
Z6 1080p FF ----------- 8.0 ms (slashcam.de)
NX500 2.5k ------------ 8.6 ms (8.2-8.8-8.9)
a7 III 1080 FF 25p ---- 8.7 ms (8.9-8.5)
a99 II 1080p ---------- 8.7 ms (8.8-8.7)
BM MicroStudio 4K ----- 9.9 ms (official)
a7r II 1080p S35 ----- 10.5 ms (11.0-10.0)
a7r II 1080p FF ------ 10.8 ms (11.0-10.5)
C700 FF S35crop ------ 10.9 ms (provideocoalition)
1DX II 1080p 24 ------ 11.1 ms (11.1-11.2)
GH5s C4k 24/30 ------- 11.3 ms (Syntheyes)
GH5s 1080 30 --------- 11.5 ms (Syntheyes)
GH5s 4k 24/30/60 ----- 11.9 ms (Syntheyes)
C700 FF widescreen --- 11.9 ms (provideocoalition)
a6000 1080p24&30 ----- 12.2 ms (13.9-11.1-11.5)
BM UMP 12k - 12k 2.4 - 12.3 ms (official)
XT3 1080p24 ---------- 12.6 ms (12.6-12.7)
BM 4.6K (4K crop) ---- 12.6 ms (official)
BMMCC ---------------- 13.3 ms (official) my measurement was 13.4 ms (13.7-13.5-12.9)
GH4 1080p ------------ 13.7 ms (13.7-13.5-13.1-14.0-13.9-13.5-13.5-13.9)
RX100 V 1080p -------- 13.9 ms (14.4-14.0---14.2-13.1) (first 2 with stabilization, last 2 without)
D750 ----------------- 14.5 ms (14.6-14.1-14.6-14.8)
1DX II 4k 24&60 ------ 14.7 ms (14.6-14.9-14.9-14.6)
RX10 ----------------- 14.8 ms (14.5-14.8-15.0)
GH5 ------------------ 15.0 ms (official, preproduction)
Z6 4k APS-C ---------- 15.0 ms (slashcam.de)
BM 4.6K -------------- 15.2 ms (official) my measurement was 16.3 ms (16.3-15.9-16.8)
a6300 1080p24 -------- 15.2 ms (15.8-14.6)
Pentax k1 1080p24 ---- 15.2 ms (15.9-15.7-13.9)
GX85 1080p ----------- 15.4 ms (13.8-16.7-15.7)
GH3 ------------------ 15.5 ms (15.4-15.7-15.4)
BM UMP 12k - 12k ----- 15.5 ms (official)
C700 FF -------------- 15.7 ms (provideocoalition)
a7R III 4k S35 ------- 16.0 ms (15.6-16.4)
XT3 4k 60fps --------- 16.0 ms (15.6-16.0-16.3)
C200 (4k=1080p) ------ 16.1 ms (4k: 15.5-16.2-16.9, 1080p: 15.5-16.0-16.6)
BM p4k 4k ------------ 16.2 ms (official)
BM p6k 5.7k ---------- 16.3 ms (official)
a7III 4k S35 25p ----- 16.6 ms (16.6)
a7III 4k 1.2x 30p ---- 16.7 ms (dpreview)
G7 1080p ------------- 16.9 ms (17.5-16.3)
XT2 1080p ------------ 16.9 ms (16.9-16.0-17.4-17.3)
RX100 IV 1080p ------- 16.9 ms (16.5-17.3) (without stabilization it's slightly faster: 16.1)
RX100 VI 4k ---------- 17.4 ms (17.4)
RX100 V 4k ----------- 17.5 ms (18.1-16.3-17.7---17.7-16.9-18.5) (first 3 with stabilization, last 3 without)
BMPCC ---------------- 17.7 ms (official) my measurement was 17.8 ms (17.7-17.7-20.0-20.0)
a7s APS-C 1080p ------ 19.5 ms (20.3-18.4-19.5-19.5)
BM p6k 6k ------------ 19.7 ms (official)
a7R II 4K FF --------- 19.9 ms (19.3-19.6-19.4-21.2)
5D3 ------------------ 20.5 ms (20.7-20.5-20.4)
XT3 4k 24fps --------- 20.6 ms (20.9-20.4-20.4)
D5200 ---------------- 22.4 ms (22.5-22.1-22.6)
GH4 4K/UHD ----------- 22.0 ms (official) my measurement was 22.5 ms(23.2-22.3-22.7-22.7-22.4-21.8-22.4-22.8-22.6)
a7R III 4k FF -------- 22.8 ms (22.6-22.9)
Z6 4k FF ------------- 23.0 ms (slashcam.de)
a99 II 4k ------------ 23.4 ms (23.3-23.5)
BMC ------------------ 23.6 ms (official) my measurement was 25.0 ms (26.7-24.8-23.5)
GX9 4k 25p ----------- 24.3 ms (24.0-24.7)
a7 III 4k FF 25p ----- 24.4 ms (24.7-24.5-23.7-24.6-24.4)
a7 III 4k FF 24p ----- 25.0 ms (dpreview)
5D2 ------------------ 25.9 ms (25.5-26.4-25.8)
5Dsr ----------------- 27.7 ms (27.5-27.9-27.6)
G7 4k ---------------- 28.1 ms (27.8-28.5)
GX85 4k -------------- 28.9 ms (30.2-29.0-28.8-27.5)
NEX-5N --------------- 29.4 ms (28.8-29.6-28.9-29.8-29.8-29.1-29.7)
Z7 4k FF ------------- 29.5 ms (30.0-29.0)
a7s II FF 1080p ------ 30.3 ms (29.5-31.2)
a7s II FF 4K --------- 30.4 ms (30.2-30.6)
a7s FF 1080p --------- 30.5 ms (30.1-32.0-30.5-30.3-29.2-30.9)
XT2 4k --------------- 30.7 ms (30.5-30.7-31.2-31.2-30.8-29.9)
X-H1 4k 24p ---------- 30.9 ms (30.4-31.3)
NX1 4K --------------- 30.9 ms (30.6-31.6-31.4-30.7-30.2)
NX1 UHD -------------- 32.6 ms (32.9-32.0-32.9-32.5)
a7R II 4K S35 -------- 33.3 ms (35.6-32.2-32.8-35.3-32.6-31.1)
RX100 IV 4K ---------- 36.6 ms (36.4-36.7) (without stabilization it's slightly faster: 35.7)
a6300 4K 24fps ------- 39.0 ms (39.2-38.9)

OTHER NUMBERS, MOST OF THEM FOUND BY SQUIG SOMEWHERE ON THE WEB:

Film --------------- 5 ms? (here)
Scarlet ----------- 14 ms
F65 --------------- 14 ms
AF100 ------------- 14.85 ms
FS100 ------------- 15 ms
FS7 --------------- 15 ms (here)
C300 -------------- 16 ms
Red One MX -------- 16.6 ms
7D ---------------- 21 ms
5D MKII ----------- 25 ms
GH1 --------------- 25 ms
D90 --------------- 33 ms

 
Last edited:
I ran the tests with my NEX-5N, and the results are very consistent... and also very different from the measures I got when I was using standard footage and trying to measure this. If the line is not vertical, the left-to-right and the right-to-left measures will be very different. Averaging them out gives a very accurate approximation of the true value, but if you don't do that (and it has to be the same vertical line, with the camera exactly in the same position), the numbers can be very misleading.

My original measure for the NEX-5N was 23.4ms, calculated with some standard footage that happened to have a half-useful pan. Then I ran the tests as described above, 5 times, with the following results: 28.8ms, 29.6ms, 28.9ms, 29.8ms, 29.8ms.
 
Last edited:
I ran the tests on a BMC. The results were not as consistent as with the 5N, but I would be happy with my overall average: 25ms (shooting RAW at 24p)
 
Last edited:
I have the US version, but yes, I was very surprised to see it compare so badly with the BMC at 24p. I never thought of this little toy as particularly shutter-impaired, maybe I've subconsciously adjusted to slow camera movement over the last couple of years, with a T2i first and then with the 5N...

Squig is sending me some files soon, so we'll have a few more cameras to compare.
 
Thanks a lot to Squig, for he sent me tests with 4 cameras!!!

NEX-5N: 29.4 (exactly the number I had with my tests!)
D5200: 22.4
5D2: 25.9
5D3: 20.5




He also sent me these numbers, which he found somewhere on the net, and which, given our 5D2 result, may be comparable with the list above:
Scarlet 14ms
F65 14ms
FS100 15ms
AF100 14.85ms
C300 16ms
Red One MX 16.6ms
7D 21ms
5D MKII 25ms
GH1 25ms
D90 33ms
 
Last edited:
look at the vertical line that gets skewed
measure horizontal difference from top point in frame t to bottom point in frame t (A pixels)
measure horizontal difference from top point in frame t to top point in frame t+1 (B pixels)

if B pixels is 1000/24=41.7ms, then A pixels is (41.7*A/B) ms

it is normal that the left pan and the right pan will give you different numbers, but the average should be pretty consistent

If you don't get it working, send me the file and I'll measure it
 
The D5200 did better than I expected, if you hadhold it the roller shutter seems worse because it's so light, same goes for the Nex-5n. But if you handhold any DSLR you're a bloody idiot!

It would be great if we could get some numbers for the GH3, GH2, 1DC, 1DX, and D800 too.
 
squig said:
The D5200 did better than I expected, if you hadhold it the roller shutter seems worse because it's so light, same goes for the Nex-5n. But if you handhold any DSLR you're a bloody idiot!

It would be great if we could get some numbers for the GH3, GH2, 1DC, 1DX, and D800 too.
Click to expand...

Add F3 and F5 to the list.
You don't need to add the F55. :evil:
 
This is the backdrop I used for the tests. I tried an outdoor scene first but with all the detail and high contrast the codec just turned the pans to mud and I couldn't even see the bloody pole!

rs test.jpg


Mods could you please sticky this thread.
 
New result, again thanks to squig, who is on a quest to collect every budget camera in the market...

GH3: 15.5ms (three measures: 15.4, 15.7, 15.4)

That puts it clearly ahead of the DSLR pack, with a shutter that is as fast as those from the video and cine cameras. Impressive. Pity about the DR, which still seems a lot worse than the competition.
 
Samuel H said:
New result, again thanks to squig, who is on a quest to collect every budget camera in the market...

GH3: 15.5ms (three measures: 15.4, 15.7, 15.4)

That puts it clearly ahead of the DSLR pack, with a shutter that is as fast as those from the video and cine cameras. Impressive. Pity about the DR, which still seems a lot worse than the competition.
Click to expand...

That's quite a respectable showing for the GH3. I presume that this is in normal mode and not ETC? ETC mode on the GH2 was much better than normal w.r.t. rolling shutter. I would expect the same to be true of the GH3.
 
On the neutral setting with everything dialled down to -5, Samuel measured it at 9.3 stops. I've done some real world comparisons to the 5D MKIII and D5200 and it's apparent that the GH3 has less dynamic range. The GH3 noise level/pattern is much better than the 5D and D5200 up to 1600 ISO, there's no fixed pattern noise.

The saga continues.
 
^ that

I wouldn't have posted that "9.3" number, since we're not sure yet about what the optimal settings are for the GH3
but now that it's online... this is the chart with those settings (in thirds of a stop):
gh3_test1.jpg
 
thanks. I would have guessed 1-2 stops more than GH2 by eyeball but have never measured (I'm not that technical minded).

I have found that the GH3 has much more recoverable shadows than the GH2.
 
Ok, so there is obviously no way to test this yet, unless John Brawley or another BMD rep decides to submit a sample, but because I'm bored at the moment, I'm going to make a prediction on the performance of the BMC-Pocket. JB has stated a couple of times that the chip in the Pocket is an S16, 1080p version of the ~2.5k chip in the BMC mk1. Knowing that the number of active photosites on the Pocket sensor is about 64% of that of its big brother, and assuming all other things being equal, we would expect a readout time of about 64% of the BMC sensor as well. That would put the Pocket at 15.6 ms, within the ballpark of the GH3 and dedicated video cameras. Not bad at all, really.

Of course, this in only informed speculation. It may not be that simple, and I could be totally wrong. The one sample video we've seen had a decent amount of movement, and no perceptible jello, so I bet it's not that far off. I'm curious to see how close to the truth that 15.6 ms number will be.


Update: Looks like I was off. My estimate falls short by 2.2ms.
 
Last edited:
I joined dvxuser pretty much so that I could say THANK YOU for putting this thread together. Jello has been so much a part of the DSLR video experience, it's incredible in a way that no-one thought to comprehensively measure it before now. I love how people are reverse engineering the cameras to reveal the really important stuff the manufacturers won't tell us. These results show you can't necessarily make assumptions based on sensor size, or the previous implementations of a given manufacturer.

I see a lot of jello in some of my NEX-5N footage. I always thought it was a combination of my crappy technique, use of non-stabilized vintage lenses, refusal to use any kind of stabilization, and copious coffee intake. Now I know it's actually the camera's fault :cheesy:

Seriously though, I'm thinking of getting a D5200, just to have something that's slightly better balanced with Nikkor lenses than using 5N+adaptor. And according to the results here, the D5200 has around a third less rolling shutter than the 5N. I tried putting my lenses on a friend's D5100, and I liked the ergonomics a lot better.

Interesting that jello on the 5N is less at 60fps than at 24fps. Anyone have any guesses as to whether any other settings on the camera could affect the amount of rolling shutter?
 
You're welcome :)

AFAIK, the only settings that will affect rolling shutter are things like fps and cropped view and such. Basically, read-out speed is a very deep characteristic of the sensor, there's not much you can do to make it faster, except read it in a completely different way.
 
You must log in or register to reply here.
Back
Top