"Deskew" proposed opensource Rolling shutter correction software

it was just a test to see if it was possible, not a workflow

But even on a tripod people move up and down, cars get distorted, this fix would not only resolve the wobble, would resolve all other inframe rolling shutter distortions too. It wasnt possible until this idea came along
 
it says it only works in host apps that support 16 bits but i'm using motion which does, and it's set up that way too...

/matt
 
so just to clarify, what's happening here is that you:

1) identify the fixed formula giving CMOS delay effects for each pixel

then, for each frame, you:

2) retrieve the coordinate change for each pixel since previous frame from the twixtor-calculated motion vector
3) apply the fixed transformation for this pixel location based on 1)

-- (then the next frame based on the corrected values?)

so the hard work is in 2), being done by the twixtor motion vector calculations (via block motion compensation?), and the accuracy depends on how well they cope with the pre-correction CMOS deformation when following shapes?

thus, we could experiment with various motion compensation algorithms (so long as we could extract the motion vectors per-pixel) and expect to see various degrees of efficacy?
 
Last edited:
Exactly. I think tracking each point one frame further to calculate acceleration then applying a bilinear correction would be fine. The time delay is fixed and only has to be calculated once. The hard part is probably where to find the missing pixels at the edges during pans and when objects move across backgrounds (blending? Forward compensation?). Not even twixtor's motion compensator succeeds with that in all situations. /matt
 
It´s worse than that...

If the cam pans in one direction and some objects are moving in another direction (that matt wrote) it´s going to look like crap. Twixtor can´t fix that automatically, you have to give twixtor mattes and stuff like that so it knows what is what. And if you have to fix that for each clip it´s not fun anymore.
And twixtors frames is not allways perfect :-) Sometimes you can slo things alot and with great results but sometimes not at all. You get wobbly smeary edges. Remember "kais power tools" :-) ?

What I´m saying is that all this work, but for now (sadly) - just in theory.
 
i'm starting to wonder if the best bet isn't simply to use a global correction, stabilizer style, optionally letting it vary linearly from top to bottom. after all the main artifact we're after is the jello, and that's caused by camera movement not subject movement. deshaker for virtualdub does a great job of exporting global motion vectors in a simple format, and it even logs the difference due to a rolling shutter. sure, this will seriously screw up the geometry of objects that move across the frame in a different direction from the pan, but then again isn't that already the case?

/matt
 
probably worth the effort to try all options. sensor cycles are 18-24mo, and it would require substantial engineering to add many more read-out channels or adopt a live MOS/global shutter approach; I doubt it could be fixed by the next revision, unless they anticipated this level of interest.
 
maybe is a stupid question, but why there are no Rolling Shutter artifacts while taking photos?

the SLR photographic shutter is mechanical but I'm guessing that's just part of explanation, because whatever the exposition is there are no CMOS scanning artifacts

Apart of the mechanical vs electrical shutter, there are different CMOS behaviors between Photo and Video?
 
an electronic shutter works by resetting the sensor, line by line, waiting the required shutter time, then reading it out, line by line. since you can only read one line at a time the next line is still being exposed while you read the current one. the cmos retains the image after it's been exposed though, so you with a mechanical shutter you reset the sensor, open the shutter, close it, then read it out and voila, no rolling shutter. (mechanical shutters roll too though, just much much faster)

/matt
 
I agree with Lee that it is a constant delay and the difference in the skew has to do with the speed at which objects pass the censor either running by a static camera, or static objects in front of a quick pan, or both camera and object in motion.

However, this can not be fixed with a time displacement filter because the image information is not present in the previous frames. The censor delay is obviously very miniscule amounts of time. And by telling the computer to take the bottom part of the previous frames does not mean it will line up and create a non-skewed frame. We are only shooting 24 or 30 fps in most cases not 720fps so that we can have a 720 line displacement for each frame.

I loaded up a clip in FCP and just tried to make a skewed tree trunk straight again by using mattes and layering the same clip on top of itself over and over. The reality is that by displacing (even every line of 720 lines) simply cannot work because the information is not there. The previous frames are just more skewed imagery.

The only way to fix a rolling shutter problem would be by a faster sensor or a complicated bit of software that can somehow analyze and adjust the skew of each frame and that sounds just about impossible.

Sorry, I am not a programmer, but I was intrigued by this older thread as I am about to get a D90 and was looking into stuff about the rolling shutters.

i enjoyed reading the thread, thanks guys.
 
You must log in or register to reply here.
Back
Top