Frame rate and jello

ydgmdlu

Veteran
[Note: This is cross-posted on the 5D forum as it also would be of keen interest to readers there.]

The rumored explanation for the withholding of the long-rumored 24p firmware for the 5D Mark II is that testers and engineers felt that the jello was too severe in the 24p mode for Canon to deem acceptable for release. I know that there are many who can't make sense of this claim, but it makes perfect sense to me. It all has to do with how the camera is recording video.

The video comes from the live view stream. So the flaws in the video should have a lot to do with how the live view stream is achieved. Each frame in the stream comes from a single pass of the electronic rolling shutter down the sensor. If the frame rate of the stream is 30 fps, then each pass of the rolling shutter takes presumably around 1/30th of a second to complete (probably a just a hair less to give the DSP time to scale and compress the image). That's not very fast at all, which is why jello appears so easily.

It could be made faster, but I'm guessing that Canon's engineers didn't feel that live view needed a faster read-refresh rate. After all, live view was only intended for previewing a still photo shot, and the 1080p video was a hack made on top of it. The sensors on dedicated camcorders are most likely engineered to have maximum read-refresh rates, specifically because proper camcorders aren't supposed to have jello, as much as it can be helped.

On the DSLRs, the different frame rates are probably achieved by simply retiming the rolling shutter's (read-refresh) clock. So for 24 fps, the clock is slowed to, say, 24 Hz (~1/24th second read time), and for 60 fps (~1/60th second read time), the clock is sped up to 60 Hz. That would be the easiest, least technically challenging thing to do. Based on this hypothesis, jello should increase or decrease as the frame rate decreases or increases (respectively). We have real-world proof of this: On the GH1, rolling shutter artifacts are virtually imperceptible in the 60p mode but much worse in the 24p and 30p modes.

The GH1 has a noticeably better rolling shutter all around versus the 5D, so the timing might be something like 48 Hz for 24p and 120 Hz for 60p. I'll bet that the main reason why the 5D has less jello than the D90 is because it's 30 fps (30 Hz) versus 24 fps (24 Hz). So if Canon does release the 24p firmware, the jello would probably be just as bad as the D90's. And this is likely the rumored technical reason why the firmware hasn't been released.

That brings us to the 7D. When the first sample videos were released, there were conflicting views on the jello; indeed, the jello seemed to vary from clip to clip. It was about as bad as the 5D on some clips, but in the fast motion clips in "Line of Sight," the jello was missing. Quite possibly, this is an indication that the 7D's rolling shutter (and clock) behaves a lot like the GH1's. In fact, I predict that that is exactly what we'll find once we get our hands on the camera.

So at the end of the day, and perhaps to state the obvious, the DSLRs' sensors just weren't designed for video. They're improving with every new generation, but we still have to ground our expectations. So if the 5D receives a 24p firmware, and if the jello proves as bad as the D90's, would you still find the 24p mode to be worthwhile?
 
Each frame in the stream comes from a single pass of the electronic rolling shutter down the sensor. If the frame rate of the stream is 30 fps, then each pass of the rolling shutter takes presumably around 1/30th of a second to complete (probably a just a hair less to give the DSP time to scale and compress the image). That's not very fast at all, which is why jello appears so easily.

I'm no engineer - but your theory doesn't seem to take shutter speed into account.
Wouldn't most people shoot 30fps at a 1/60th shutter?

And what about shooting at much higher shutter speeds, like 1/500th or higher?
 
The "shutter speed" (i.e. exposure time) and the actual rolling shutter work separately. Think of it this way: A rolling shutter exposes the sensor one line at a time. The "shutter speed" is how fast each line is being exposed. But the amount of time that the shutter allocates to each line, before moving to the next line, stays constant.

If exposure time were intimately tied with the rolling shutter's clock, then shooting at a high shutter speed would theoretically reduce jello. But this is not the case. The Jello remains constant regardless of shutter speed, but the motion blur changes accordingly.

By the way, the numbers that I gave above are just for the sake of argument. The rolling shutter in the 5D might actually take 1/60th of a second to complete a single scan, or it might be some other number, but the idea is that the clock would adjust according to frame rate. And it's probably a multiple of the frame rate.
 
Last edited:
Think of it this way: A rolling shutter exposes the sensor one line at a time. The "shutter speed" is how fast each line is being exposed.
Gotcha - good explanation.

By the way, the numbers that I gave above are just for the sake of argument. The rolling shutter in the 5D might actually take 1/60th of a second to complete a single scan, or it might be some other number, but the idea is that the clock would adjust according to frame rate. And it's probably a multiple of the frame rate.
Makes sense to me....
:thumbsup:
 
So you suggesting that if you give sensor more time to capture frame then it screws the footage more and more cause in 60p it must work much faster than in 24p which is less than half - makes no sense cause 24p should be less demanding for sensor,but also it could be true.Who knows how engineers programmed this thing.
Anyway whats the difference,unless someone can fix the sensor we have what we have and we have to deal with it.
 
So you suggesting that if you give sensor more time to capture frame then it screws the footage more and more cause in 60p it must work much faster than in 24p which is less than half - makes no sense cause 24p should be less demanding for sensor,but also it could be true.Who knows how engineers programmed this thing.
Anyway whats the difference,unless someone can fix the sensor we have what we have and we have to deal with it.
I'm saying that it has almost nothing to do with how "demanding" it is on the sensor. I gave the reasons why the Canon engineers might've decided to do live view and video in the way that I described. It's all about engineering shortcuts and hacks. After all, they weren't trying to create a proper camcorder.

How else would you explain the fact that rolling shutter on the GH1 is a non-issue in 60p but very apparent in 24p and 30p? How else would you explain the fact that rolling shutter on the 7D is a non-issue in some high motion clips (like in "Line of Sight") but very apparent in other clips with less motion?
 
i would describe it as....
....We cant do anything about it anyway.

BUT how about telling sensor GET 60 frames but capture only 24 ? With proper timing to avoid jerkiness.
 
How else would you explain the fact that rolling shutter on the GH1 is a non-issue in 60p but very apparent in 24p and 30p?

Well, I wouldn't say it's 'very apparent' ... but definitely more apparent.
But now I'm just splitting hairs - I see what you're saying.

How else would you explain the fact that rolling shutter on the 7D is a non-issue in some high motion clips (like in "Line of Sight") but very apparent in other clips with less motion?

I think it's probably too early to judge the rolling shutter on the 7D just yet given that we haven't seen enough from it - but you bring up some interesting points.

It isn't simply DSLRs either - the same could be said of the HPX300, the rolling shutter skew was reviewed as being most noticeable at 24, and significantly less at higher framerates (which makes sense).
 
If the sensor refresh was tied to the shutter speed (which would make more sense) it should be possible to reduce jello by increasing shutter speed. Has this been tested on the GH1 or 5d2? It would make for more "stuttery" motion though (the opening of Saving Private Ryan being an extreme example).
 
Another question for you all: How else would you explain how a CMOS camera, whether it's a DSLR or a real camcorder (like the HPX300, as Luis mentioned above), derives its frame rate from the sensor scan rate?
 
If the sensor refresh was tied to the shutter speed (which would make more sense) it should be possible to reduce jello by increasing shutter speed.
Sensor scan rate is tied to frame rate (but not always). So you can have 24Hz or 48Hz scan rate for 24p, 30Hz or 60Hz for 30p, 60hz for 60i, etc. It is obvious that the faster scan rate the less jello. This is why on the HPX300 for example the jello/skew is most noticeable at 24p and is barely noticeable at 60i. I had the Sony HC1 and at 60i I did not notice any skew/jello when panning left right in totally "home video" manner.
 
If the sensor refresh was tied to the shutter speed (which would make more sense) it should be possible to reduce jello by increasing shutter speed.
It doesn't make sense for two reasons:

1. You can shoot video at 1/500th of a second on the 5D. If shutter speed and sensor scan rate were tied together, then that means that the sensor is being scanned ten times faster than it is at 1/50th of a second. That's just ridiculous.

2. We've had the D90 and the 5D Mark II for a year now. If faster shutter speeds were really the way to reduce jello, then somebody would've discovered it by now. Furthermore, since the D90 still relies upon auto-exposure in video mode, it is in control of the shutter speed. If you watch a lot of D90 videos, you'll notice that many of the ones shot in bright sunlight are stuttery, because the D90 had to use a fast shutter speed. Yet the horrible, characteristic jello is still there.
 
By the way, I remembered reading a thread on cinema5D.com in which somebody actually determined the sensor scan rate of the 5D. Turns out it actually is 1/60th of a second. I took a moment to find that thread again; here it is: http://www.cinema5d.com/viewtopic.php?f=14&t=1332

So it looks like we have the digital equivalent of the mechanical rolling shutter on film motion picture cameras, in the following sense: The 5D takes 1/60th of a second to scan the sensor, then it uses the remaining 1/60th of a second of "darkness" to do all of the necessary processing. For 24p, it would be working at 48 Hz, analogous to the 1/48th second exposure time for film.
 
Last edited:
So how is it that the 24fps on the GH1 better than the D90 in regards to jello? Is this the sensor refresh rate that Cranky mentioned? Is this a selectable option (apart from frame rate and shutter speed)? Sorry in advance for being an ignoramus but I don't have any of these cameras. Was thinking of eventually getting a GH1 but now am leaning towards 7d.
 
Jello on the GH1 at 24p is much better than the D90. I think that it's also better than the 5D's 30p.

Sensor scan rate is a fixed aspect of the hardware. It can't be changed.
 
Interesting thought, but the fact is that it's not that bad on the 7d. Go back and look at Perya - you can see minor skew in some shots, but check out the sequence with the guy who has the blinkenlight display, and the shot immediately before it (defocused strobing light spill over a guy cooking).

You can see horizontal rolling-shutter artifacts if you go through it frame by frame (eg sometimes the strobe light goes off mid-scan and instead of a pleasing round focus blur the light is just gone from the rest of the frame. But they're really not that severe. On the blinkenlights shot there's 10 or 12 different flashing light sources, some stationary and some being spun around on a string, which should bring out the worst...but it looks fine.

Not saying the issue doesn't exist, but it doesn't look like a deal breaker either compared to some other cameras.
 
By the way, I remembered reading a thread on cinema5D.com in which somebody actually determined the sensor scan rate of the 5D. Turns out it actually is 1/60th of a second. I took a moment to find that thread again; here it is: http://www.cinema5d.com/viewtopic.php?f=14&t=1332

So it looks like we have the digital equivalent of the mechanical rolling shutter on film motion picture cameras, in the following sense: The 5D takes 1/60th of a second to scan the sensor, then it uses the remaining 1/60th of a second of "darkness" to do all of the necessary processing. For 24p, it would be working at 48 Hz, analogous to the 1/48th second exposure time for film.
Mmmm... so the sensor scan rate is dependant on frame rate... so it would appear it is in some sense selectable...

Is there any reason why it couldn't select a scan rate of 60Hz for 24fps which would give it the same amount of skew/jello as for 60fps... because (for the GH1 abd 7D at least) the sensor is capable of the faster scanning?
 
If the sensor refresh was tied to the shutter speed (which would make more sense) it should be possible to reduce jello by increasing shutter speed.
Sensor refresh rate is completely unrelated to shutter speed. Shutter speed has NO effect on jello whatsoever. This has been tested and proven over and over. Shutter speed affects blur, but not jello. The amount of skew is identical regardless of shutter speed, the only thing that changes is how crisp the edges of the leaning objects are.
 
Is there any reason why it couldn't select a scan rate of 60Hz for 24fps which would give it the same amount of skew/jello as for 60fps... because (for the GH1 abd 7D at least) the sensor is capable of the faster scanning?
Heat, battery life, and physical limitations of how quickly the chip can be scanned.

Canon already has a warning in their manual saying that extended recording can cause a drop in video quality (due to overheating). Running the chip faster than the minimum necessary speed will result in more heat generated.

Your idea is solid; this is what the Thomson Infinity does, and I suspect strongly the EX1 and GH1 do: scanning at double frame rate. But those chips were apparently designed to handle it, whereas the 7D and 5D appear not to have been. But, it's still largely unknown with the 7D, so we'll see.
 
Back
Top