Announcement

Collapse
No announcement yet.

Measuring rolling shutter: put a number on this issue!

Collapse
This is a sticky topic.
X
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

    Measuring rolling shutter: put a number on this issue!

    So, in the D5200 thread here, we're kind of worried that this exceptional little toy may have more rolling shutter issues than other cameras.

    This should be a measurable thing: if we knew how many ms it takes each camera to read the sensor, we wouldn't have to be looking at all those piles of not-really-comparable youtube tests.
    Also, when a new camera comes out, we would just have to measure it: no need to compare the D5200 first against a GH2, then against a D800, then a 5D3, then...

    In order to get a half-decent measure of this, I would need to get a very short clip from each camera, shot following these instructions:
    * camera on a tripod, well levelled, set at 24p and 1/50s shutter
    * a vertical line somewhere in the frame, crossing it completely from top to bottom (it can be a pole, or a corner, or a door, etc; if it is clearly tilted it will distort my measure)
    * start recording, do a quick pan left at constant speed, then a quick pan right at constant speed, repeat, reapeat, then stop recording
    (that's a total of 3 pans left and 3 pans right, which will help average off the measurement errors)
    (to get a constant speed, do a long pan that goes on for a while, even if it has to be slower, instead of just a quick whip-and-stop; if you can't find two consecutive frames where the skew looks the same, then the pan is not good)

    If you want to know how your camera fares in relation to others, record that, then send me a link to the unedited file to similaar.feedback@gmail.com (or through a PM on dvxuser) and I'll use two consecutive frames to measure top-to-bottom delay in relation to frame-to-frame delay.

    EDIT, because lots of people write to ask how I measure this:

    First I calculate the horizontal frame-to-frame displacement, e.g. at the top of the frame. Say that's 90 pixels. That means the same portion of the image is moving 90 pixels every 42 ms (actually, 1000/24).
    Then I go to the first of those two frames and measure the horizontal displacement within the frame, from top to bottom of that vertical line. Say that's 60 pixels, which, at 90 pixels every 42ms, means a first rolling-shutter measure of 28 ms.
    Same with the second of those two frames. Say that's 70 pixels; our second RS measurement is 32ms. This means that the camera is accelerating a bit, that's why it's important to measure both frames: there's an error, but it will be averaged out (this is a first-order approximation: if the acceleration itself is accelerating, then that second-order error won't be averaged out, but in most cases that's going to be a very small factor).
    Then look for another pair of frames where the image is moving in the opposite direction, and repeat those three measurements.
    Say our measurements are: 105 for the inter-frame displacement, and 65 and 75 for the intra-frame ones, which gives us measures of 26 and 30 ms. So, the camera was faster this time but we don't really care because we're translating everything into time units; it was accelerating again this time, but as before that's not a (big) problem; and we also know that the vertical line was not perfectly vertical, but that's why we take measures in both directions, again this errors will be averaged out.
    So, the average of those four numbers is 29ms.
    I repeat everything again two or thee times more, because manually measuring blurry images is not an exact science, and that's how I get the final number that I post here.





    BM UMP2 2k ------------ 3.2 ms (official)
    RX100 V 250 fps ------- 3.8 ms (3.8-3.8)
    BM UMP 12k - 4k crop -- 3.9 ms (official)
    C700 FF s16 crop ------ 5.2 ms (provideocoalition)
    a7R III 1080 S35 ------ 6.1 ms (6.6-5.7)
    BM UMP 12k - 8k 2.4 --- 6.1 ms (official)
    BM 4.6K (2K crop)------ 6.3 ms (official)
    BM UMP2 4k ------------ 6.3 ms (official)
    1DX II 1080p 60&120 --- 6.7 ms (6.6-6.7)
    XT3 1080p 120fps ------ 7.4 ms (7.4-7.5)
    BM UMP2 4.6k ---------- 7.6 ms (official)
    a7R III 1080 FF ------- 7.7 ms (8.7-6.8)
    BM p4k HD crop -------- 7.8 ms (official)
    BM UMP 12k - 6k crop -- 7.8 ms (official)
    BM UMP 12k - 8k/4k ---- 7.8 ms (official)
    NX1 1080p ------------- 7.9 ms (7.7-8.0-8.1-7.8)
    BM p6k 2.7k crop ------ 8.0 ms (official)
    Z6 1080p FF ----------- 8.0 ms (slashcam.de)
    NX500 2.5k ------------ 8.6 ms (8.2-8.8-8.9)
    a7 III 1080 FF 25p ---- 8.7 ms (8.9-8.5)
    a99 II 1080p ---------- 8.7 ms (8.8-8.7)
    BM MicroStudio 4K ----- 9.9 ms (official)
    a7r II 1080p S35 ----- 10.5 ms (11.0-10.0)
    a7r II 1080p FF ------ 10.8 ms (11.0-10.5)
    C700 FF S35crop ------ 10.9 ms (provideocoalition)
    1DX II 1080p 24 ------ 11.1 ms (11.1-11.2)
    GH5s C4k 24/30 ------- 11.3 ms (Syntheyes)
    GH5s 1080 30 --------- 11.5 ms (Syntheyes)
    GH5s 4k 24/30/60 ----- 11.9 ms (Syntheyes)
    C700 FF widescreen --- 11.9 ms (provideocoalition)
    a6000 1080p24&30 ----- 12.2 ms (13.9-11.1-11.5)
    BM UMP 12k - 12k 2.4 - 12.3 ms (official)
    XT3 1080p24 ---------- 12.6 ms (12.6-12.7)
    BM 4.6K (4K crop) ---- 12.6 ms (official)
    BMMCC ---------------- 13.3 ms (official) my measurement was 13.4 ms (13.7-13.5-12.9)
    GH4 1080p ------------ 13.7 ms (13.7-13.5-13.1-14.0-13.9-13.5-13.5-13.9)
    RX100 V 1080p -------- 13.9 ms (14.4-14.0---14.2-13.1) (first 2 with stabilization, last 2 without)
    D750 ----------------- 14.5 ms (14.6-14.1-14.6-14.8)
    1DX II 4k 24&60 ------ 14.7 ms (14.6-14.9-14.9-14.6)
    RX10 ----------------- 14.8 ms (14.5-14.8-15.0)
    GH5 ------------------ 15.0 ms (official, preproduction)
    Z6 4k APS-C ---------- 15.0 ms (slashcam.de)
    BM 4.6K -------------- 15.2 ms (official) my measurement was 16.3 ms (16.3-15.9-16.8)
    a6300 1080p24 -------- 15.2 ms (15.8-14.6)
    Pentax k1 1080p24 ---- 15.2 ms (15.9-15.7-13.9)
    GX85 1080p ----------- 15.4 ms (13.8-16.7-15.7)
    GH3 ------------------ 15.5 ms (15.4-15.7-15.4)
    BM UMP 12k - 12k ----- 15.5 ms (official)
    C700 FF -------------- 15.7 ms (provideocoalition)
    a7R III 4k S35 ------- 16.0 ms (15.6-16.4)
    XT3 4k 60fps --------- 16.0 ms (15.6-16.0-16.3)
    C200 (4k=1080p) ------ 16.1 ms (4k: 15.5-16.2-16.9, 1080p: 15.5-16.0-16.6)
    BM p4k 4k ------------ 16.2 ms (official)
    BM p6k 5.7k ---------- 16.3 ms (official)
    a7III 4k S35 25p ----- 16.6 ms (16.6)
    a7III 4k 1.2x 30p ---- 16.7 ms (dpreview)
    G7 1080p ------------- 16.9 ms (17.5-16.3)
    XT2 1080p ------------ 16.9 ms (16.9-16.0-17.4-17.3)
    RX100 IV 1080p ------- 16.9 ms (16.5-17.3) (without stabilization it's slightly faster: 16.1)
    RX100 VI 4k ---------- 17.4 ms (17.4)
    RX100 V 4k ----------- 17.5 ms (18.1-16.3-17.7---17.7-16.9-18.5) (first 3 with stabilization, last 3 without)
    BMPCC ---------------- 17.7 ms (official) my measurement was 17.8 ms (17.7-17.7-20.0-20.0)
    a7s APS-C 1080p ------ 19.5 ms (20.3-18.4-19.5-19.5)
    BM p6k 6k ------------ 19.7 ms (official)
    a7R II 4K FF --------- 19.9 ms (19.3-19.6-19.4-21.2)
    5D3 ------------------ 20.5 ms (20.7-20.5-20.4)
    XT3 4k 24fps --------- 20.6 ms (20.9-20.4-20.4)
    D5200 ---------------- 22.4 ms (22.5-22.1-22.6)
    GH4 4K/UHD ----------- 22.0 ms (official) my measurement was 22.5 ms(23.2-22.3-22.7-22.7-22.4-21.8-22.4-22.8-22.6)
    a7R III 4k FF -------- 22.8 ms (22.6-22.9)
    Z6 4k FF ------------- 23.0 ms (slashcam.de)
    a99 II 4k ------------ 23.4 ms (23.3-23.5)
    BMC ------------------ 23.6 ms (official) my measurement was 25.0 ms (26.7-24.8-23.5)
    GX9 4k 25p ----------- 24.3 ms (24.0-24.7)
    a7 III 4k FF 25p ----- 24.4 ms (24.7-24.5-23.7-24.6-24.4)
    a7 III 4k FF 24p ----- 25.0 ms (dpreview)
    5D2 ------------------ 25.9 ms (25.5-26.4-25.8)
    5Dsr ----------------- 27.7 ms (27.5-27.9-27.6)
    G7 4k ---------------- 28.1 ms (27.8-28.5)
    GX85 4k -------------- 28.9 ms (30.2-29.0-28.8-27.5)
    NEX-5N --------------- 29.4 ms (28.8-29.6-28.9-29.8-29.8-29.1-29.7)
    Z7 4k FF ------------- 29.5 ms (30.0-29.0)
    a7s II FF 1080p ------ 30.3 ms (29.5-31.2)
    a7s II FF 4K --------- 30.4 ms (30.2-30.6)
    a7s FF 1080p --------- 30.5 ms (30.1-32.0-30.5-30.3-29.2-30.9)
    XT2 4k --------------- 30.7 ms (30.5-30.7-31.2-31.2-30.8-29.9)
    X-H1 4k 24p ---------- 30.9 ms (30.4-31.3)
    NX1 4K --------------- 30.9 ms (30.6-31.6-31.4-30.7-30.2)
    NX1 UHD -------------- 32.6 ms (32.9-32.0-32.9-32.5)
    a7R II 4K S35 -------- 33.3 ms (35.6-32.2-32.8-35.3-32.6-31.1)
    RX100 IV 4K ---------- 36.6 ms (36.4-36.7) (without stabilization it's slightly faster: 35.7)
    a6300 4K 24fps ------- 39.0 ms (39.2-38.9)

    OTHER NUMBERS, MOST OF THEM FOUND BY SQUIG SOMEWHERE ON THE WEB:

    Film --------------- 5 ms? (here)
    Scarlet ----------- 14 ms
    F65 --------------- 14 ms
    AF100 ------------- 14.85 ms
    FS100 ------------- 15 ms
    FS7 --------------- 15 ms (here)
    C300 -------------- 16 ms
    Red One MX -------- 16.6 ms
    7D ---------------- 21 ms
    5D MKII ----------- 25 ms
    GH1 --------------- 25 ms
    D90 --------------- 33 ms

    Last edited by Samuel H; 09-22-2020, 01:41 AM.

    #2
    I ran the tests with my NEX-5N, and the results are very consistent... and also very different from the measures I got when I was using standard footage and trying to measure this. If the line is not vertical, the left-to-right and the right-to-left measures will be very different. Averaging them out gives a very accurate approximation of the true value, but if you don't do that (and it has to be the same vertical line, with the camera exactly in the same position), the numbers can be very misleading.

    My original measure for the NEX-5N was 23.4ms, calculated with some standard footage that happened to have a half-useful pan. Then I ran the tests as described above, 5 times, with the following results: 28.8ms, 29.6ms, 28.9ms, 29.8ms, 29.8ms.
    Last edited by Samuel H; 03-20-2013, 10:31 AM.

    Comment


      #3
      I ran the tests on a BMC. The results were not as consistent as with the 5N, but I would be happy with my overall average: 25ms (shooting RAW at 24p)
      Last edited by Samuel H; 03-19-2013, 03:16 AM.

      Comment


        #4
        5N has a different rolling shutter depending on if it's 50p or 25p. In 25p it's quite bad.

        Comment


          #5
          I have the US version, but yes, I was very surprised to see it compare so badly with the BMC at 24p. I never thought of this little toy as particularly shutter-impaired, maybe I've subconsciously adjusted to slow camera movement over the last couple of years, with a T2i first and then with the 5N...

          Squig is sending me some files soon, so we'll have a few more cameras to compare.

          Comment


            #6
            Thanks a lot to Squig, for he sent me tests with 4 cameras!!!

            NEX-5N: 29.4 (exactly the number I had with my tests!)
            D5200: 22.4
            5D2: 25.9
            5D3: 20.5




            He also sent me these numbers, which he found somewhere on the net, and which, given our 5D2 result, may be comparable with the list above:
            Scarlet 14ms
            F65 14ms
            FS100 15ms
            AF100 14.85ms
            C300 16ms
            Red One MX 16.6ms
            7D 21ms
            5D MKII 25ms
            GH1 25ms
            D90 33ms
            Last edited by Samuel H; 03-20-2013, 10:21 AM.

            Comment


              #7
              How do you get these numbers?

              I have a feeling my RX100 is pretty good here, but no 24p

              Comment


                #8
                look at the vertical line that gets skewed
                measure horizontal difference from top point in frame t to bottom point in frame t (A pixels)
                measure horizontal difference from top point in frame t to top point in frame t+1 (B pixels)

                if B pixels is 1000/24=41.7ms, then A pixels is (41.7*A/B) ms

                it is normal that the left pan and the right pan will give you different numbers, but the average should be pretty consistent

                If you don't get it working, send me the file and I'll measure it

                Comment


                  #9
                  The D5200 did better than I expected, if you hadhold it the roller shutter seems worse because it's so light, same goes for the Nex-5n. But if you handhold any DSLR you're a bloody idiot!

                  It would be great if we could get some numbers for the GH3, GH2, 1DC, 1DX, and D800 too.
                  gamma-movie.com The Indie Gathering 2016 Sci-Fi feature screenplay award winner.

                  Comment


                    #10
                    Originally posted by squig View Post
                    The D5200 did better than I expected, if you hadhold it the roller shutter seems worse because it's so light, same goes for the Nex-5n. But if you handhold any DSLR you're a bloody idiot!

                    It would be great if we could get some numbers for the GH3, GH2, 1DC, 1DX, and D800 too.
                    Add F3 and F5 to the list.
                    You don't need to add the F55.

                    Comment


                      #11
                      This is the backdrop I used for the tests. I tried an outdoor scene first but with all the detail and high contrast the codec just turned the pans to mud and I couldn't even see the bloody pole!

                      rs test.jpg


                      Mods could you please sticky this thread.
                      gamma-movie.com The Indie Gathering 2016 Sci-Fi feature screenplay award winner.

                      Comment


                        #12
                        New result, again thanks to squig, who is on a quest to collect every budget camera in the market...

                        GH3: 15.5ms (three measures: 15.4, 15.7, 15.4)

                        That puts it clearly ahead of the DSLR pack, with a shutter that is as fast as those from the video and cine cameras. Impressive. Pity about the DR, which still seems a lot worse than the competition.

                        Comment


                          #13
                          Originally posted by Samuel H View Post
                          New result, again thanks to squig, who is on a quest to collect every budget camera in the market...

                          GH3: 15.5ms (three measures: 15.4, 15.7, 15.4)

                          That puts it clearly ahead of the DSLR pack, with a shutter that is as fast as those from the video and cine cameras. Impressive. Pity about the DR, which still seems a lot worse than the competition.
                          That's quite a respectable showing for the GH3. I presume that this is in normal mode and not ETC? ETC mode on the GH2 was much better than normal w.r.t. rolling shutter. I would expect the same to be true of the GH3.
                          ...............KABOOOOMM!!!: https://vimeo.com/38327387

                          Comment


                            #14
                            Originally posted by Samuel H View Post
                            Pity about the DR, which still seems a lot worse than the competition.
                            But still much improved over the GH2

                            What is the DR measurement for the GH3? I only have gone by my eye and it looks a lot better than GH2 and on par with 5D2 but that is just my eye.

                            Comment


                              #15
                              On the neutral setting with everything dialled down to -5, Samuel measured it at 9.3 stops. I've done some real world comparisons to the 5D MKIII and D5200 and it's apparent that the GH3 has less dynamic range. The GH3 noise level/pattern is much better than the 5D and D5200 up to 1600 ISO, there's no fixed pattern noise.

                              The saga continues.
                              gamma-movie.com The Indie Gathering 2016 Sci-Fi feature screenplay award winner.

                              Comment

                              Working...
                              X