D90 Stair-Stepping Removal

Interesting -

...i'll be getting a D90 soon (mainly for stills though), so this inspired me to make my own version, using a displacement map. It has only one layer plus one tga image to deal with, instead of those sliced layers. My method looks more or less as good (as far as i can tell), but probably renders quite a bit faster (didn't test the original version).

Here's the AEP (v 7.0), in case you're interested ;-)

http://eki.pp.fi/digivideo/D90/D90FIX.zip

The same .tga file should work pretty much universally, with any app that has displacement map functionality. In AE, setting of 0.3 in vertical displacement gave decent results - this will vary between applications, of course. I tried with Vegas - it works, so i added the .veg file to the zip.


D90_FIX_Demo.jpg
 
Last edited:
is it just me, or is this a way easier solution?
combine this displacement map with the noise thing from neatvideo?
seems like you guys (lee & matt, too) would create the nikon d91x (whatever, when they "fix" their issues."
roll in an anti-jello filter and she-bang - awesomeness for <$2000 !!!
 
Cool, keep the ideas coming. Of course this is a much easier method but as I'm in the After Effects Club since yesterday I just tried it and I find while it looks v-e-e-e-ry good it doesn't look quite as good as Dr. Wilsons cure. As you can see there are still some tiny stairsteps. But maybe thats not important on real world images (especially when you downscale to 360p). Halsu, is it possible to fine tune the displacement map to get the same smooth edges as in the right most image?

1224319305.gif


How did you make the map? I mean the tga file. I noticed that you used an uniform 8 pixel distance and that the black line isn't always black. What happens when the displacement map pushes pixels on top of each other? Become they interpolated?
 
Last edited:
Halsu said:
Interesting -

...i'll be getting a D90 soon (mainly for stills though), so this inspired me to make my own version, using a displacement map. It has only one layer plus one tga image to deal with, instead of those sliced layers. My method looks more or less as good (as far as i can tell), but probably renders quite a bit faster (didn't test the original version).

Here's the AEP (v 7.0), in case you're interested ;-)

http://eki.pp.fi/digivideo/D90/D90FIX.zip

The same .tga file should work pretty much universally, with any app that has displacement map functionality. In AE, setting of 0.3 in vertical displacement gave decent results - this will vary between applications, of course. I tried with Vegas - it works, so i added the .veg file to the zip.
Click to expand...



Brilliant !

It doesn't give quite a good result as actually going in there and chopping every single step out - but it is 80-90% as good and for a whole lot less effort and more transferable to other software.

Genius ! :)
 
What bugs me about all this, is: Why couldn't nikon have done this! I mean, is this really the easiest way? Get crappy video files and suffer to bring them back to life in post? Couldn't some of this (noise reduction, line interpolation) be done in hardware by the camera? Jeez...

But nice touch with the displacement map!
 
buildyo said:
How did you make the map? I mean the tga file. I noticed that you used an uniform 8 pixel distance and that the black line isn't always black.
Click to expand...

I made a 9 pixel tall gradient strip in photoshop using the standard gradient tool, then copied that in after effects, eyeballing it so that it matched the jaggies. When i tried this with the AE displacement plugin, it resulted in pretty weird bulges on each strip.

I applied a gamma correction to the displacement map, again eyeballing the correction that gave straight lines - the resulting gamma was 2.2, i recall, which probably has much to do with video's default 2.22 gamma.

The places where the black lines show as gray are places where the strips weren't exactly 9 pixels apart - i didn't make a separate 8 pixel wide strip, positioning the 9 pixel strip half pixel off looked OK with my eyeball method... true 8 pixel strips might have been better, but i was a bit tired at the time - it was 2 am local time ;-)

The results most probably could be further improved by adjusting the displacement map, i might have a go at it some time later.

What happens when the displacement map pushes pixels on top of each other? Become they interpolated?
Click to expand...
My aim was basically to scale each 9 pixel strip vertically (black areas in the disp map push pixels up, white areas push them down). I think the pixels that get pushed together more or less disappear, they get scaled so small while being pushed from two sides... but i am not sure of this really, it's all just guessing ;-).
 
Last edited:
taubkin said:
What bugs me about all this, is: Why couldn't nikon have done this! I mean, is this really the easiest way? Get crappy video files and suffer to bring them back to life in post? Couldn't some of this (noise reduction, line interpolation) be done in hardware by the camera? Jeez...

But nice touch with the displacement map!
Click to expand...

I'm pretty sure it all boils down to processing power, which boils down to price... If my dual core laptop's processors can't do it in real time, no wonder if a camera's built in one cant.
 
Halsu, you did a great job eyeballing this!
I wrote an Illustrator script to fill the individual 7px, 8px and 9px slices with gray values according to your procedure pasted it into photoshop and saved it as a tif file. I think it's now closer to the LEEFIX but I'm biased :-) It would help if we could shift the map in finer steps than 0.1.

1224355463.gif


Right-click the link below to download my displacement map:
http://www.dvxuser6.com/uploaded/39976/1224345291.tif
 
Last edited:
Halsu said:
Excellent!!
Click to expand...

Thanks a lot for this method Halsu. It is not only simpler it also seems to soften JPEG compression artefacts. Only disadvantage so far is that the very first and last pixel row stands out.

The mandatory Face Test:
1224363539.jpg


... also thanks to Nikon. Without your botch, I wouldn't have learned of clever slicing and interpolating by shifting layers or the existency of a thing called displacement map. :beer:
 
Last edited:
One last comparison. I don't see any difference to the result of Lee's method:
http://www.dvxuser6.com/uploaded/39976/1224418288.jpg

A displacement map doesn't alter pixel values. It just pushes them around. There's no interpolation and the original pixels are still there just not the ones at the Jaggies. Do I understand this correctly?

Halsu, is your name Erkki Hannes Ilmari Halkka? You also deserve a statue! :beer:
 
Last edited:
Using the Displacement Map in Final Cut Pro 6

It works the stair steps become smaller, but for some reasons the map file works a lot better in After Effects.

Right-click the link below and download the displacement map:
http://www.dvxuser6.com/uploaded/39976/1224345291.tif

Import the displacement map TIFF file into FCP

Apply Effects > Video Filters > Distort > Displace to a D90 clip

Open the Filters panel and drag the displacement map on the Map Image icon

Set
Horizontal Scale 0
Vertical Scale 0.15
Check Repeat Edge
 
Last edited:
My only question about this is if it's still okay to do in post sharpening and adjustment without bringing back the jaggies too much. For instance, buildyo, your picture of the little boy looks better after the displacement map fix but, if I had an image like that, I would want to add just a little extra sharpening in FCP, say, around +10 to +20 depending. Would this be okay?
 
Jack_Felis said:
My only question about this is if it's still okay to do in post sharpening and adjustment without bringing back the jaggies too much. For instance, buildyo, your picture of the little boy looks better after the displacement map fix but, if I had an image like that, I would want to add just a little extra sharpening in FCP, say, around +10 to +20 depending. Would this be okay?
Click to expand...

If you sharpen the displacement filtered clip some jaggies come back but if you sharpen a rendered copy they don't come back.
 
Last edited:
buildyo said:
Using the Displacement Map in Final Cut Pro
-edit-
Set
Horizontal Scale 0
Vertical Scale 0,15
Check Repeat Edge
Click to expand...

I'm having a little trouble with this in FCE. If I try to enter 0,15 in the vertical scale box, it states I must make a selection first. Is this right?

BTW pretty great problem solving guys...

One other thing, I notice that the stair step becomes visible on the back panel LCD only after the OK button is pressed.
 
mdifazio said:
I'm having a little trouble with this in FCE. If I try to enter 0,15 in the vertical scale box, it states I must make a selection first. Is this right?
Click to expand...
You must double click a clip to load it into the viewer, then apply the displacement effect.


mdifazio said:
One other thing, I notice that the stair step becomes visible on the back panel LCD only after the OK button is pressed.
Click to expand...
I also noticed that. It shows that Nikon uses faster, less quality downscaling when it records movies.
 
You must log in or register to reply here.
Back
Top