Clever, FREE pulldown-removal & transcoding workflow via AviSynth

this pulldown removal issue on the PC is driving me insane as to which is the best and what the losses are.. can someone give a complete list of various ways from free to expensive to do it correctly?
 
Blackout said:
this pulldown removal issue on the PC is driving me insane as to which is the best and what the losses are.. can someone give a complete list of various ways from free to expensive to do it correctly?
Click to expand...


Doesn't get much easier than NeoScene: http://www.cineform.com/neoscene/

It's fast, it's stupid easy, and the fact it removes the pulldown is just a side benefit. The best benefit is it transcodes to 4:2:2 Cineform AVI...which you can edit on a Commodore 64. Even Sony Vegas chokes on AVCHD on my quad-core PC...AVC simply sucks for editing.

Cons: It cost money...but so does the GH1.
Pros: It makes me money...but so does the GH1.
 
Blackout said:
this pulldown removal issue on the PC is driving me insane as to which is the best and what the losses are.. can someone give a complete list of various ways from free to expensive to do it correctly?
Click to expand...

Actually, algorithm-wise, there is only one way... And there is no magic behind it. As to pixel format, with free approach you can choose your output pixel format and compression as you like.

Look, I know that once we are in the land of post-processing you have to pay money for software, and it seems the more you pay the better software becomes.
In the case of pulldown removal, there is only one algorithm. Point. Does not matter how much money you pay, the output quality re pulldown will be the same for proper removal.

This is the algorithm:

x0 = selectevery(10, 0,1).Weave()
x1 = selectevery(10, 2,3).Weave()
x2 = selectevery(10, 5,6).Weave()
x3 = selectevery(10, 8,9).Weave()

It makes 4 frames out of 10 fields, picking the right fields to build each progressive frame.

To use that, one needs to install:http://avisynth.org/
1. Avisynth
2. ffdshow tryouts
3. Haali Media Splitter
4. VirtualDub

NOTE: Win 7 users must disable built-in mp4 decoder for this to work.

Then cut the below code and paste it into .bat file i.e. 24p.bat
Launch this file in the folder with your .mts files, it will process all files that are 1920x1080 and make 1080p .avi file for each corresponding .mts file.

Note that in the .bat below compression is commented out and output format set to 11 that corresponds to 4:2:2 YCbCr (YUY2)

You can set these values to your liking. To find out what value corresponds to what, open virtualdub, set Video->Color Depth (or/and Compression), hit Ctrl-S, open saved file, locate SetOutputFormat and/or SetCompression, paste corresponding values to your batch file.

The file below does not set compression and sets pixel format to to 4:2:2 YCbCr (YUY2)

---- 24p.bat begin ----

for %%f IN (*.mts) DO (
echo directshowsource^("%%f"^).AssumeTFF^(^).SeparateFields^(^) > %%~nf.avs
echo Assert^(width^(^) == 1920, "Not a 1080p24 source"^) >> %%~nf.avs
echo x0 = selectevery^(10, 0,1^).Weave^(^) >> %%~nf.avs
echo x1 = selectevery^(10, 2,3^).Weave^(^) >> %%~nf.avs
echo x2 = selectevery^(10, 5,6^).Weave^(^) >> %%~nf.avs
echo x3 = selectevery^(10, 8,9^).Weave^(^) >> %%~nf.avs

echo Interleave^(x0,x1,x2,x3^) >> %%~nf.avs
)

for %%f IN (*.avs) DO (
echo VirtualDub.Open^("%%f"^); > %%~nf.script
REM echo VirtualDub.video.SetCompression^(0x636c6c63,0,10000,0^); >> %%~nf.script
echo VirtualDub.video.SetOutputFormat^(11^); >> %%~nf.script
echo VirtualDub.SaveAVI^("%%~nf.avi"^); >> %%~nf.script
echo VirtualDub.Close^(^); >> %%~nf.script
)

for %%f IN (*.script) DO (
"C:\Program Files\VirtualDub\vdub.exe" /min /x /s %%~ff
)

for %%f IN (*.script) DO (
del /Q %%f
)

for %%f IN (*.avs) DO (
del /Q %%f
)

---- 24p.bat end ---
 
Not to bring up a dead thread but none of these methods work. Is there still no solution for PC other than neoscene and its bloated output?
 
When you deinterlace and then reconstruct 24 real frames from a couple I frames and a bunch of P frames you don't expect your file size to get bigger?
 
Not 500x bigger, no. Doesnt matter anyways, voltaicHD crashes, neoscene is too bloated and expensive to use, and this avisynth method produces combing artifacts and astronomically large files. I'd shoot native but the camera crashes if i so much as point it outdoors.

edit: mutabot's script works! Its the scripts at the beginning of the thread that dont.
 
Last edited:
tonpole,

NeoScene will produces 24p output between 70-120Mb/s based on the quality setting you choose, using a high-quality I-frame wavelet. Long GOP AVCHD in the GH1 is 17Mb/s (unless you hack the firmware for a higher bit-rate.) So the data size increase is 4 to 7X, and is well worth it (in you hack the camera in increase is even less.) The datarate is equilivent to other I-frame intermediate codec, such as Avid's DNxHD or Apple's ProRES. With all intermediate codecs, the video is much easier to edit, another reason only $99 for NeoScene goes a long way to help tens of thousands of users.
 
David,

Does using the hack, which produces bigger MTS files, also produce bigger files out of NeoScene? I assume it does not (or, at least not much) because the NeoScene output is already much higher bitrates than AVCHD. Certainly if one choses all I frames it seems that it shouldn't make much difference.

Chris
 
No, it will not produce noticabley larger files in NeoScene, in some cases you might get smaller files. The artifacts of AVCHD can produce false detail that CineForm has to compress. The datarate on the CineForm output is image dependent, a clean simple image will be lower data rate that a complex noisy one.
 
mojo43 said:
Has anyone else tried mutabot's way of removing pulldown with success?
Click to expand...


Yes, but it requires a few changes, so here's the updated script:


Code: 
---- 24p.bat begin ----

for %%f IN (*.mts) DO (
echo directshowsource^("%%f"^).AssumeTFF^(^).SeparateFields^(^) > %%~nf.avs
echo Assert^(width^(^) == 1920, "Not a 1080p24 source"^) >> %%~nf.avs
echo x0 = selectevery^(10, 0,1^).Weave^(^) >> %%~nf.avs
echo x1 = selectevery^(10, 2,3^).Weave^(^) >> %%~nf.avs
echo x2 = selectevery^(10, 5,6^).Weave^(^) >> %%~nf.avs
echo x3 = selectevery^(10, 8,9^).Weave^(^) >> %%~nf.avs

echo Interleave^(x0,x1,x2,x3^) >> %%~nf.avs
)

for %%f IN (*.avs) DO (
echo VirtualDub.Open^("%%f"^); >> %%~nf.script
echo VirtualDub.video.SetCompression^("Lags",0,10000,0^); >> %%~nf.script 
echo VirtualDub.video.SetOutputFormat^(11^); >> %%~nf.script
echo VirtualDub.SaveAVI^("%%~nf.avi"^); >> %%~nf.script
echo VirtualDub.Close^(^); >> %%~nf.script
)

for %%f IN (*.script) DO (
"C:\Program Files\VirtualDub\vdub.exe" /min /x /s %%~ff   
)

for %%f IN (*.script) DO (
del /Q %%f
)

for %%f IN (*.avs) DO (
del /Q %%f
)

---- 24p.bat end ---
A Few notes:

1. Make sure the vdub location is set correctly.

2. I set this to encode to lagarith so you can get lossless output at decent size. You can set this to whatever you want, just put the fourcc of whatever codec you want to use in the quotes, and this is case sensitive so try a few variations if it doesnt work. Also make sure your codec is set to output YUY2.

3. Put this in the same folder as your files and it will encode all of them automatically. Dont leave any spaces in any of the folder names in the hierarchy or it wont work.


//
 
Fantastic!!! It looks great...Thanks a lot tonpole!

I can't see any artifacting, are you finding any problems with this way of removing pulldown?
 
Ive compared it side by side to a locked scene using the native patch and its identical, no weird combing artifacts or chroma bleeds like with the other methods.
 
Ya, I am not seeing anything either. I need to do some more tests, but all seems to look correct.

Here you go... a free pulldown removal method!! I slapped another loop in the batch script and did all of my folders on one go. Easy as pie!

Thanks...
 
This is a really nice find. This makes it possible to not have to worry about the Native 24p issue so much - which may not have a solution any time soon.

Nice work!

Chris
 
Does Tonpole's updated script also work for 25P (50i wrapper) source? if not , what shoud be revised? I wanna pure 25p source ready before it is imported into NLE .
 
You'd have to ask the author but hes apparently hasnt been on in months. If it doesnt work you can always go over to doom9's avisynth forum and ask them to modify it for 25p.
 
I revised the script to output 25p for pal user.

---- 25p.bat begin
for %%f IN (*.mts) DO (
echo directshowsource^("%%f"^).AssumeTFF^(^).SeparateFields^(^) > %%~nf.avs
echo Assert^(width^(^) == 1920, "Not a 1080p25 source"^) >> %%~nf.avs
echo x0 = selectevery^(2, 0,1^).Weave^(^) >> %%~nf.avs

echo Interleave^(x0^) >> %%~nf.avs
)

for %%f IN (*.avs) DO (
echo VirtualDub.Open^("%%f"^); >> %%~nf.script
echo VirtualDub.video.SetCompression^("Lags",0,10000,0^); >> %%~nf.script
echo VirtualDub.video.SetOutputFormat^(11^); >> %%~nf.script
echo VirtualDub.SaveAVI^("%%~nf.avi"^); >> %%~nf.script
echo VirtualDub.Close^(^); >> %%~nf.script
)

for %%f IN (*.script) DO (
"C:\VirtualDub-1.9.8\vdub.exe" /min /x /s %%~ff
)

for %%f IN (*.script) DO (
del /Q %%f
)

for %%f IN (*.avs) DO (
del /Q %%f
)

---- 25p.bat end
Click to expand...

Please help test it and correct me if I am wrong .

but , another problem arise : I put the oringinal mts file and pulldowned avi file into premiere cs3 and watched in the y/c waveform monitor. since the Lag codec considered to be lossless , the 2 files 's y/c waveform should be identical . but the result is NOT . the avi file has slightly narrower luminance width than the original mts file . I guess this is caused by 0-255 to 16-235 convertion . Just a guess . My question is "how to avoid this ?" . I want to losslessness convertion .
 
You must log in or register to reply here.
Back
Top