Nikon D90 Stair-step and Rolling Shutter fix batch script

graphfruit

graphfruit

Member
I'm developing a batch program for Nikon D90 (and the upcoming D5000) movie clips which will remove the "stair-stepping" and minimize the rolling shutter lag (Nikon are you listening?!?), outputing a lossless video file for further post-processing.
Currently the main features are:

Nikon D90 Stair-step and Rolling Shutter fix batch script
​
- Batch processing of all video files in the working catalog
- Stair-stepping removal acc. to Lee Wilson's fix
- Specific rolling shutter fix - (yet to be optimized)
- (Optional) stabilisation to eliminate shakiness when hand-held
- Output in lossless (Huffyuv) video codec - (yet to be optimized)

Test 1 - Before and after:
http://www.youtube.com/watch?v=yLL46767KqY


My batch script is currently in an early testing phase and still needs quite a lot of optimization.

Could anyone contribute in the development by sending me some wobbly video clips,allow me to further optimize my settings?

My Nikon D90 is currently on service so I don't have enough video material to further test my script. Please upload the original footage w/o any postprocessing using a filehosting service such as RapidShare. I'll test my script on your video clips and I'll send you the fixed clips back to you so you also can evaluate the result.

Due to the way the rolling shutter behaves on the D90 it's not possible to fully eliminate the problem, but at least minimize the problem mostly when panning horizontally. On the other hand fixing Nikon's terrible scaling technique, that produces "stair-steps", works very well (huge thanks to Lee Wilson and other dvxusers).

I've prepared a simple test chart so it would be great if someone could print it out (A3 or A4), set up your camera on a tripod and put the chart up on a wall in a bright place and do some panning back and forth at different speeds, preferable with a zoom lens. Also then some up and down seperately and some diagonal movement.

Test chart 1:
http://www.graphfruit.com/wp-content...graphfruit.gif

// Adrian Graphfruit
http://www.graphfruit.com/scripts/nikon-d90/
 
Last edited:
Oh my Odin. The Swedes will save us. Will upload the jagged jello on rapidshare or wherever and send you details.
 
I'm just adding some info that my batch script should also work with video clips from the new Nikon D5000, as it also suffers from the same severe rolling shutter. Probably some settings will need to be adjusted, time will tell. Looks like Nikon put in the same sensor as in the D90.

To prove my point that the (pre-production) D5000 have the same problem:
"The movies from our pre-production unit look an awful lot like those we got from the D90, oddly enough." (http://www.dpreview.com/previews/nikond5000/page10.asp)

"Live View and HD video recording. We've confirmed both are identical implementations to the D90 before it." (http://www.cameralabs.com/reviews/Ni.../verdict.shtml)

The "jello" due to the rolling shutter is visible in these commercials:
http://www.youtube.com/watch?v=TF7P-Q4fMcc
http://www.youtube.com/watch?v=NeChp9FDEFU

And IRL video:
http://www.youtube.com/watch?v=24fa4rnPaiY
 
Last edited:
Once done could this script also be used on any other type of cam footage (outside of D90 and D5000) that also exhibits CMOS skew?
 
@Ian-T
Yes of course, but the roller shutter fix is using a program that is already out. What's unique with my approach is that I'm trying optimize specific settings (looks like every second frame needs a different correction) for the harsh wobbling in D90 videos and offer more of a bundle program that will correct all files in a batch and spit out lossless files.
 
what software are you using? i'm experimenting with, also swedish btw, plugin deshaker which works well for stabilizing rolling shutter wobble as part of the "deshaking" (i measured it to 98% btw). i'd love to see a piece of software that does it without actually stabilizing, shouldn't be hard to write once the basic algorithms are in place.

/matt
 
grahfruit:

how about render times? Will it take a long time for the AVI's to go thru the batch process?
Also...what is the quality output? Does it degrade th original by much?
If you need me to shoot specific examples for you to use, just ask...i will be happy too.
 
@mattsand

I'm actually also using deshaker (all good is coming from Sweden? Nikon should have been Swedish) but with other settings (and delays etc) and removed the stabilisation (thought of adding it as an optional setting). My main point is to make it simple to minimize the D90 problems w/o using VirtualDub and w/o users intervention with different settings and running in 2 passes and then redoing this for every video clip. Start the batch and take a break while it does its thing.

My initial idea was to make a plugin for Vegas (similar to yours, Matt) with the addition of deshaker, but the mandatory 1:st pass for movement detection and slow speed wouldn't make it possible to achieve this. That's why I see a lossless-coded output files as a second best thing.
 
@Super8
At this early stage, the batch is quite slow but that's mainly because I'm using the highest movement detection settings in order to establish optimal correction settings.

The degradation (if any or visible) depends mostly on output codec (the deskewing of the picture may in theory also affect this). As I'm aming for getting output files for later post-processing in your editing program of choice, my goal is to maintain as much quality as possible, even the file gets bigger than the original file. As the batch program is built up at this stage, the audio is left untouched.

I thought of outputting in Huffyuv. Do you have a better suggestion?

Thanks for offering help with shooting example videos. I'm not sure if I can count on getting a new/repaired Nikon D90 anytime soon.

I've prepared a simple test chart so it would be great if someone could print it out (A3 or A4), set up your camera on a tripod and put the chart up on a wall in a bright place and do some panning back and forth at different speeds, preferable with a zoom lens. Also then some up and down seperately and some diagonal movement.

Test chart 1:
http://www.graphfruit.com/wp-content/uploads/nikond90-testchart1-graphfruit.gif
 
graphfruit said:
I'm actually also using deshaker (all good is coming from Sweden? Nikon should have been Swedish) but with other settings (and delays etc) and removed the stabilisation
Click to expand...
how does that work? i've found that if you set the amount to very little no skew is removed either, or am i wrong?

/matt
 
graphfruit said:
I thought of outputting in Huffyuv. Do you have a better suggestion?
Click to expand...
mjpeg at max quality and 4:4:4 will probably produce similar results at a fraction of the bitrate. not only does this save disk space, it also edits better if you decide to do the post as a pre. but huffyuv is great.

/matt
 
In my tests i've made so far it skews (corrects) the picture very well when panning even if motion smoothness is off (=0). Here's an example of how it's correcting even if motion smoothness is off:
http://www.graphfruit.com/wp-content/uploads/nikond90-example-skew.jpg

The frames doesn't move in the center part (where the red line goes) while the upper and lower part is skewed, giving a "wider" image but also introducing black bars. The black parts gets later on filled in with similar frames so I won't need to zoom into the video.

The script is currently already outputing in mjpeg right now so I'll just need to choose better quality settings.
 
Currently I've got it narrowed down to around 77-80%. That's what I'm still trying to optimize. I was pixel-peeping for a couple of my test shots and noticed a pattern that every second frame the frame gets skewed differently (hypothetically due to unsynced "rolling shutter", but there's other stuff to take into consideration), so you'll need a different percentage now and then. With 98% I saw that the correction was too strong. That's why I need some clean video footage with that test chart to better see what's happening.
 
I also tried his method of calculating the angle but in practice the frames were skewed too much using my video files. I ended up pixel-peeping and a lot trial-and-error. Try using your value of 98%, but 0 for all in motion smoothing in 2ndpass, take one still frame and one frame during movement, open in Photoshop, align them manually and compare with the ruler tool.

There's also a small "jump" when a panning stops, but that's something I'll try to approach once the frames are straight in the middle of the panning.
 
i've done some testing now, and 98% works really well. in fact i think i'm gonna run almost my entire feature through deshaker. we managed to avoid most of the jello and i don't really mind slanted lines, but there's often a slight wobble to the image when you change direction in the panning that can be annoying, and this software removes it completely, you don't notice any difference except the frame looks more "dead" afterwards. and yes you're right it does work even with stabiization set to zero. i don't want any stabilization, just jello removal. maybe we can convince gunnar to write a separate script for that? it shouldn't really need a first pass either since in that particular case it only needs to analyze a couple of frames at a time.

/matt
 
You must log in or register to reply here.
Back
Top