Recommended Tools - Sorry guys, Windows Only!

JeganRX

JeganRX

Well-known member
Encoding is unfortunately one of those dark water topics that very few people seem willing to dive into; I can't blame anyone, there's a LOT of information to take in at once, and it really just takes years of practice, memorization, and developing an extremely fine eye that can scrutinize any video to pieces.

But there are a number of issues that people here seem to face on a daily basis, so I would like to explain, in as simplistic terms as possible (avoiding such things as color space coefficients, the exact graphical descriptions of telecine cadence patterns, etc, instead relying on more general, intuitive forms of these topics). I tried to create a thread almost a year ago on how to get around most encoding obstacles for free, but I never got around to finishing it, so for this one, I'll be more brief, and just list software solutions. It'll be up to you to research them.

Part 1 - Tools That Everyone Should Download and Get Used To:

x264 (StaxRip) - Undisputed king of H.264 encoding. I'm not a fan of the command line myself (though the addition of presets has made it significantly easier to encode with), so I recommend StaxRip for almost all H.264 purposes. It includes presets for many mobile devices, standalone devices, and of course, customization over every x264 parameter. MPEG-4 ASP (Divx, Xvid, etc) is also covered here if you wish, and the program can convert most audio formats for you. It has built in filters for most basic processing functions, such as deinterlacing, inverse telecine, resizing, denoising, etc, though I'll touch on that later, since I prefer to do it externally myself.

AVISynth - A requirement of Staxrip actually, AVISynth is an extremely powerful scripting language to call video, audio, images, etc, and filter them in almost any variety of ways. You can significantly, and very effectively reduce noise in your videos, you can deinterlace better than any commercial NLE can, and you can even perform proper IVTC of 24p-in-60i footage. You can do colorspace and color sampling conversions, proper interlaced scaling, and yes, even editing. I'm not a master of the language myself, but many of the best filters and scripts are easy to learn. Trust me, it can get scary, but only for the functions that some of the crazies on Doom9.org create; you'll likely never want to spend much time a lot of the slow deinterlacing and denoising functions, though some of them are indeed godsends.

AvsP - My current preferred previewer for AVISynth scripts. Since AVISynth scripts are simply text files that "load" a video, it's useful to have a previewer. The popular, but old VirtualDub (and VirtualDubMod for its AVS editor) was my former choice, but it lacks some features that AvsP has, such as autofill for AVISynth, and slider bars that load based on your input. It features tabbed scripting for better navigation.

VirtualDub - I don't really recommend this program anymore. When I'm lazy, I'll export something in Huffyuv, Lagarith, or even Xvid out of here, since it is a relatively quick way to do a test file. Get it if you must, but otherwise stick to AvsP.

Foobar2k w/ Nero AAC output, and AVISynth input - My favorite minimalist audio player, also serves as a great audio encoder. The great part is combining the .avs input filter with it so that you can directly load your AVISynth scripts into foobar and export to any format you choose. It's great if you're cutting up something in AVISynth and want to make an audio track that adheres to it, along with giving you more control over the encode. You can also get a variety of input plugins, such as AC3, which will allow you to export editable WAVs from any AC3 file you may have. Nero AAC will come with Staxrip, though if you want, you can acquire it separately (either way, you must locate the file for foobar) from Nero's website.

___________________________

 Optional:

LAME MP3 - In reality, I don't consider this optional, but it's included with Staxrip if I'm not mistaken. These days, I rarely encode to MP3, but I do my occasional CD ripping, so it's useful to have for foobar2k.

FAVC - Contrary to the acronym, it's not an H.264 encoder, it's actually a quick DVD building script. Feed this program .avs scripts, configure a few tabs, and it'll spit out a very basic DVD quite quickly using HC or QuEnc MPEG2 encoders (it's up to you). Great for quick demo discs or dailies for clients.

Audacity - I assume that many of you use professional, commercial audio editors, but for most tasks, Audacity is really good. I prefer Audition for most edits since I find it more stable, but Audacity is great when coupled with ffdshow input, as it can read almost any format you want to throw at it. You can load AC3 files, edit them, and re-export them (this of course means a loss, but if you must, it's available).

ImgBurn - This is inclusive of FAVC I believe, but it's great to have on its own too. ImgBurn is exactly what every DVD burning program should be: Lightweight, simple, intelligent (it will detect if you're trying to burn a DVD VIDEO_TS folder, but have not set the appropriate file system settings), and highly configurable. It also makes setting dual layer break points very easy, offering you optimum points to perform layer switches. I don't even use any other burning tool.

YAMB - It's actually a GUI for MP4Box, but damn it, it's the best one around. It's literally a muxer, demuxer, and editor for .MP4 files. This is a great tool because you can extract streams from other file types (say, MKV or MOV), and place them into more friendly MP4 containers. You can create highly customized .MP4 files from this program, and of course, extract raw streams from MP4s. This is a must have for me, really, but because StaxRip can export MP4s on its own (presumably through MP4Box, though I'd have to check), I've placed it in optional.

MKVMerge - Another GUI frontend for a muxer. You guys may not have much use for MKV files, but I personally love them, as they are the ultimate containers for almost any format imaginable. You can have multiple video tracks, chapters that link to other video files, audio tracks in different formats, attached fonts for subtitles, many more subtitle formats than MP4 can ever support, and far greater support for more video formats. It's a dream container... and it's royalty free. Use this for your own purposes, such as backing up your DVDs. I use this a lot for my own collection, but it's mostly optional.


Stay tuned for Part 2 - What You Can Do With These Programs (AVISynth)... You won't be disappointed.
 
Last edited:
I use YAMB exclusvely to take h.264+AAC from .mov containers to .mp4 containers. It doesn't change the data at all, but it makes the file much more compatible (and accurate) in lots of software+hardware players. It's great for those post houses that don't know/don't care about Quicktime gamma shift.
 
Part 2 - What You Can Do With These Programs (AVISynth)

Ok, so I just made you download a bunch of software you probably haven't even heard of, or at least avoided using for some time. In my last thread, I attempted to give examples and solutions for a few common problems, in a very tiresome manner. Again, this time around, I will simply list common problems, and follow up with my recommended solutions. There WILL be code displayed here, as it's necessary for AVISynth, but I will try to point you in the direct of proper tutorials (because really, almost every program here has features that you must explore on you rown). Let's begin with the basics. You will need my AVISynth plugins folder (unless you wish to download them individually!) for these tasks.

http://www.douglasguerra.com/plugins.7z <--- AVISynth Plugins


Loading Your Video Into AVISynth
You can create a .avs script by hand in notepad, if you wish, but assuming you've downloaded AvsP, you can simplify the process as it will generate a script for you! Open up AvsP, and like most programs, use file -> open to locate your video. Now, since this is a Windows-only guide, I think it's fairly safe to assume you've exported your video into an AVI format. If not, that's ok, go back to your project and export to some lossless AVI format (Lagarith, Huffyuv, PNG etc... Google these, they're useful). Locate your video in AvsP and open it; the top portion of the window will fill with a code like this:

Code: 
AVISource("...")
"..." being your video source of course. It does require quotations, by the way. Save this script, and simply press F5 to preview the video in the window below! If the codecs are installed properly, you should have no problem viewing your video. Note: There ARE other ways of importing video to AVISynth, such as DirectShowSource, MPEG2Source, FFMPEGSource, I suggest you look these up at the AVS website).


Denoising/Degraining
Because this seems to be something people often run into, I'm covering this first. There are many filters to denoise your footage with, but I prefer Mdegrain from the wonderful MVTools2 plugin. It is included with my plugins RAR, and here's the code to use it!

Code: 
ConvertToYV12 #Only necessary if your video isn't in 4:2:0 color, which it probably isn't if you used Lagarith)
super = MSuper(last)
backward_vec2 = MAnalyse(super, isb = true, delta = 2, overlap=2)
forward_vec2 = MAnalyse(super, isb = false, delta = 2, overlap=2)
MDegrain1(last, super, backward_vec2,forward_vec2,thSAD=400)
return last
This goes right below your AVISouce() command, but just an important note: When you use the line "return last", it seems to disable any filters that you do after that line, so try to use mdegrain last. F5 your video, and it should look a good deal cleaner. Now remember, this won't make ISO3200 look like ISO100, you still need to be reasonable here, but it can make ISO800 look pretty damn close to ISO200 on my T2i's footage. Mdegrain is great in how it very organically treats the footage. It doesn't smear your noise much, creating that plastic-y look, and often times, helps with some banding issues as well.


Inverse Telecine and Deinterlacing (They're two different things!)

IVTC as it's frequently abbreviated, is exactly what it says - it's inverting the telecine process, matching the interlaced fields, and then removing the duplicate frames from 24p material that is in 60i. Occasionally, Premiere will fail to capture a DV stream as progressive for whatever reason, or you may have older footage around that is 24fps, but was never flagged for it. In any case, you can remove that with a simple plugin: "TIVTC" There are MANY parameters to this filter, and I suggest looking them, but for simply IVTC operations, I frequently just use the bare minimum call:

Code: 
TFM(pp=0)
TDecimate()
The PP parameter stands for post processing, it's intended for when TIVTC cannot correctly match the fields of the video, and must instead deinterlace that frame. This sounds desirable, but I often find it confusing fine detail with interlacing, and it horribly post processes those frames. I prefer to use a filter called "Vinverse" (also in my plugins) to remove stray fields when necessary. I use pp=0 to turn off post processing.

Now for deinterlacing, you have a number of options. These days, you should be shooting progressive all the time, but if you do have some 1080i footage, or older 480i fooage, you can deinterlace decently, and efficiently with something called "TDeint" (same guy behind TIVTC if you didn't get it). For the web, we usually stay away from 60p deinterlacing (bobbing), as most sites don't offer it, so we'll stick with 30p.

Code: 
TDeint(mode=2)
At 1080i, this may slow down a bit, but it should work very well on 480i footage. You may notice jaggies and aliasing in many areas - alas this is inevitable (though it can be helped), and while there are other deinterlacers available in my folder, I'll let you look them up... There's way too many to detail here.


Resizing and Scaling

This is the easy part of AVISynth - and it's also built-in. Bottom line is, and this is my opinion, upscaling will always look bad. I know many of you swear by your upconverting Blu-ray players, but it can never make detail that isn't there. All any hardware (or software) can do over another, is do a slightly better job. Downscaling on the other hand, has it's purpose - web video, DVD mastering, etc, although if you're a crazy AVISynth user like myself, you'll probably want to utilize a little something called "supersampling" from time to time when filtering... Anyway, you can resize video using a few different resizing algorithms, and here's my preferences from most preferred to least preferred:
Code: 
Spline16Resize(1280,720) #Width and Height are entered as such
Spline36Resize() #Sharper than Spline16, but slightly slower, so I only use it for upscaling
Lanczos4Resize() #Sharp resizer, but can lead to artifacts on the edges
LanczosResize() #Sharp resizer, but can lead to artifacts on the edges
BicubicResize() #Soft resizer, but can be configured to be close to lanczos, use it for downscaling
BilinearResize() #Softest resizer, use it for downscaling
PointResize #This is equivalent to "nearest neighbor" resize in Photoshop. It's useful for some supersampling functions.
The cool part of AVISynth is that it will respond to mathematics quite literally, so you can type 720*2,480*2, and it will in fact give you that result (1440x960). Be extra careful NOT TO RESIZE WHEN INTERLACED. YOU MUST DEINTERLACE OR IVTC BEFORE RESIZING. There are proper ways to resize when interlaced, but I'll go over a quick and simple way to do it. For example, if we wished to downscale 1080i to 480i for DVD:
Code: 
SeparateFields() #This reduces your videos vertical resolution by half, do not worry.
Spline16Resize(720,240) #The height must be half of your destination height, eg. 720x480 would be 720x240.
Weave() #Reinterlaces your fields, your height is restored.
Interlacing progressive footage. (Yes, this is possible)

From time to time, you may want to convert that 1080 or 720p60 footage to 1080i/60, or 480i/60 (DVD or broadcast if the network wants 1080i). It's really simple to do this conversion. First off, you might as well resize as you would normally, to the destination resolution (as you're starting progressive). Now place this into the script:

Code: 
Spline16Resize(720,480) #This is whatever your destination resolution should be
AssumeTFF().SeparateFields().SelectEvery(4, 0, 3).Weave()
By the way, placing periods between commands is the same as spacing them down, if you're wondering.

So that about covers the majority of functions that I imagine most DVXUsers would need. I recommend the following websites if you want to learn about many, many more (advanced) filters and functions for AVISynth. You can also convert colorspaces, color sampling, change framerates, color balance, etc.

http://forum.doom9.org
www.doom10.org
www.AVISynth.org

Next up is
Part 3: What You Can Do With These Programs (StaxRip)
 
Last edited:
Part 3 - What You Can Do With These Programs (StaxRip)

Intro:
StaxRip, as I said in the very post, is actually just a front end for the great x264 encoder. x264 of course, is the name of a freeware H.264 encoder, and I dare say, objectively the greatest H.264 encoder around. It has recently reached Blu-ray compliance, and the hope is that it can be implemented commercially (though I myself am curious how this will fly with the MPEG-LA since it's an open source project, we shall see), since quite honestly, the majority of H.264 are just wasteful in their bitrates. x264 is highly optimized, and very fine tuned; it is fast (relative to other encoders), it is highly configurable, and looks great.

If you are not in a last minute rush to deliver something, I highly recommend - if not exclusively advocate - x264 as your delivery encoder. Completely disregard Adobe's (MainConcept's actually) encoder, Apple's encoder, or just about any other solution. Hardware solutions also fail to match x264's quality. Many major video sites use a branch of x264 (YouTube notably has their own branch, and Vimeo uses a particular build as well).

Again, like before, I will do little explaining about the MANY features of x264 - which are mirrored in the GUI of StaxRip. I also choose StaxRip myself because it allows audio encoding and muxing as well, eliminating a few steps in most of my encodes. Encoding is a daunting task, and if you really want to use x264, I suggest you read up on its configurations.


http://mewiki.project357.com/wiki/X264_Settings <-- This is applicable to the Command Line and StaxRip.


Starting With StaxRip:

Since StaxRip is actually a collection of other programs, you'll want to leave its folder alone. It has a few dependencies (notably AVISynth), so make sure those are installed, though it should warn you when you open they're not present. Your beginning screen should look something like this:

staxrip1.png


Looks intimidating? I admit, I'm not a fan of StaxRip's UI (I prefer MeGUI, but the program is terribly outdated and buggy), but once I help you get around most of the features, you'll feel comfortable. What we see here, going row by row, left to right, is our source file, and the destination file, that's easy enough. Moving down, we see filters - these are based on AVISynth filters, and StaxRip tries to make AVISynth filtering a little easier for you by maintaining its own filter set. In my opinion, this just complicates matters, so I generally leave all the options unchecked (except source - automatic, which is necessary). The Resizer is also AVISynth-based; however, it's a very strict resizer. You should take the time to learn the difference between DAR, SAR, and PAR, but chances are you won't be dealing with non-square pixels anyway, if you're encoding to H.264.

Of course, that assumes you have already resized (or didn't need to) in AVISynth, BEFORE coming into StaxRip. The Error box is also important. If you decide to resize in StaxRip, the error box tells you how deviated you are from your intended aspect ratio. You can move the slider to figure it out for yourself, as it's also a quick way to find out proper resolutions for your video (proper meaning divisible by 16).


The Codecs
Over in the middle-right side, we have our codec choice. Right now, it defaults to DivX Plus (x264 in disguise), though if you click the "DivX Plus" name in blue, you'll get a wide variety of presets to choose from. There's plenty of hardware presets here too, if you're aiming for certain devices, though I myself have custom profile. You can configure the codec below, in the grey text, but do yourself a favor and leave it alone unless you've read the x264 command link from above. Right next to the codec name, you see the container choice - by default it's MKV, and knowing most of you, you'll want MP4. Most of the time, I simply choose "MP4 Container", but when I want, I use the iPhone/iPod compatible version. Do not worry, there's no real difference here when it comes to desktop playback.

Let's just step back a second, there's one setting I would like you to get to know in the "Config Codec" menu.:The Quality setting. Hit "config codec" and you'll get this screen:

staxrip2.png


Now yours will look different by default, but I've already chosen the "Devices - PS3" preset from the first screen, so it has a few different settings. The default quality setting is also 23, but I've changed it to 20. Unlike many other encoders, x264 (and other freeware encoders as well, such as Xvid) lists in quality in descending numbers. This is referred to as "Constant Rate Factor" on many websites, including the x264 web page, though StaxRip uses the term "Quality", so be careful. I suggest reading up on this if you want to know how it functions, but as a general guideline, I aim for 20, which is very good quality, for web purposes. For higher quality, or BD purposes, I would use something closer to 17 or 18. You cannot predict filesizes in CRF (Quality) mode, so if you wish to pick a desired filesize, use one of the two-pass mode presets (and look it up on the x264 page). Anyway, after selecting your quality, hit ok, and you'll return to the main screen. Mine looks like this:

staxrip3.png



Audio:
When it comes to audio, I suggest you either follow my guide for foobar2k (unwritten as of now), OR you take the immediate route and just export your audio separately from your NLE. A WAV file will be fine. Audio encoding is much more straight forward, and StaxRip has a number of presets for it. Load up your WAV file (in my case, I already have encoded an AAC file from foobar2k, but I will show you the menus anyway) and choose a preset. For most delivery purposes, AAC will be fine, but you should probably increase the bitrate over the preset's 96k. Simply hit "edit" next to the preset and the following screen will come up:

staxrip4.png


Now you can switch this into a bitrate mode if you desire, but Nero AAC (which StaxRip uses) is far more efficient in its own quality based mode, much like x264. Since the audio will be recompressed by most websites, I prefer to play it safe and go a bit high for AAC - or quality 0.50, which you will see equates to roughly 160kbps. It may end up encoding lower, it's all up to the encoder! You can save this preset if you desire, but simply hit ok to move on back to the main screen. By the way, if you ARE using a pre-encoded file like I am (be it MP3 or AAC), do not use a preset; click the blue audio preset (defaulted to "AAC VBR ~96 kbps") and choose "Just Mux".


Conclusion:
At this point, you'll be set to encode your video! Easy enough, right? It gets very easy after a few times of playing around, since you can copy your AVISynth script to each video, and save your encoding presets as well. Hit Next twice to proceed, as you must first add the encode to a batch (yes you can batch encode!). If any problems exist, mainly with the dimensions of the video, StaxRip will not let you proceed, and the warning will appear in the bottom of the screen. The jobs screen will come up, and you'll see your batch list appear. You can either close the list and queue up more encodes, or simply hit start.

The rest is just waiting. The stats of the encode will come up as its going, so have fun studying x264's behavior.

Next Up: Encoding Audio Separately - Foobar2k (It's a short one!)
 
Last edited:
If your head is still spinning, that's because compressing a file right is HARD!

When in doubt with a paying client, hire a professional. I think there might be a few in this thread right now ;)
 
Part 4 - Encoding Audio Separately (Foobar2k)

Intro:
Foobar2k is a great minimalist audio player, that uses very few resources, runs fast, and also serves as an excellent (and customizable!) audio encoder. For encoding, it actually serves as a frontend (GUI) to many command line based encoders, and yes, you can program it to correspond to the command line, but we don't have to do that, because Foobar comes with presets for AAC encoding! All you have to do is download Nero's AAC Encoder (it's free). For this tutorial, we *won't* be using the AVISynth input component, as that implies some more advanced AVISynth commands to load audio and video together, but it's good to have.

Setup:
For our sake, all we need to "install" is the AAC codec, and the link is the first post of this thread. You can put it anywhere really, foobar2k will ask for it when you try to encode AAC, so just remember where it is.

Reminder: Nero AAC comes with Staxrip, so if you wish to use that copy, go ahead.

So let's go ahead and open our audio file. I assume you're starting with uncompressed WAV files, but MP3s work, and you can also get components to read directly from AVI files (DirectShow input I believe), AC3 files, and even DTS files.

Simply right click the file (now in your playlist), go to convert, then hit the "..." option, which is usually browse on most applications. This screen will come up:

foobar1.png


Mine is preconfigured, as this is how I've last used it, but there should be a default option for 175kbps AAC. Hitting the browse button next to the "Output Format" allows you to setup the bitrate as you wish. I chose 125kbps in this example, but I do indeed use 175kbps on more archival stuff, or if it's going to be recompressed. The rest of menu is pretty easy to figure out, but it's largely insignificant unless you want to automatically label your output files. You can largely ignore the processing tab for now, although you can take a look if you want.

Hit ok when you're done, you'll first be asked for the name of the AAC file you want to save, and then you'll be asked to locate Nero AAC's encoder file. Don't worry, this is only necessary the first time. The audio should only take a few seconds to a few minutes - audio never takes long to encode. You can now use this in Staxrip if you wish!
 
And that's it! The tutorial is complete. There's a lot that I didn't cover, however, as it would take eons of explanation. What makes encoding so complicated is the sheer amount of scenarios, problems, and solutions that come up. You need to be wise about figuring out what you're dealing with, and then how to deal with it.

Feel free to PM me or ask in this thread about any specifics you may have, otherwise, good luck encoding.
 
Just found this thread. Dude: That was a monumental amount of work. Thanks for doing all the heavy lifting. I'm working through it now. I see it's for Windows, and I'm on a Mac, but I'm reading through it simply for the sake of trying to understand the processes involved themselves. Thanks again. I hope by highlighting it more people notice it.
 
All of the post houses I know use a Windows PC to compress final deliverables for 2 reasons:

1) Cheaper cost of higher end hardware.
2) Software is more supported, and you have more options.

I don't consider Episode Pro a Post-house solution. Not unless you're running SD downconverts through a piece of hardware since their scaler is BILNEAR only.
 
Demistate said:
2) Software is more supported, and you have more options.
Click to expand...

People, use your Macs for the least expected purpose: To run Windows! You should be grateful that it's much easier to dual boot Windows on a Mac, than it is to dual boot OS X on a PC, with with driver ubiquity.
 
Hi Douglas, a.k.a. RX782. Thank you for posting this thread! I'm starting to work my way through this process but I've noticed your avisynth download link:

http://www.sendspace.com/file/t7u8hm

is no longer working.

Maybe I didn't see them clearly but could you please post the plugins necessary or revive that link so I may download your plugin folder?

All of your work here is much appreciated regardless. You've helped me understand this process more.

Cheers!

- A
 
Many thanks for putting this together. I was playing around with these tools today and they most definitely output much higher quality 264 files than CS5. I also played around with uploading some samples to vimeo to see if I could improve the quality but found their specs bottlenecked the quality. The uploads did not look good at all, even though they were outstanding on my PC.

In particular, I have a short clip of tall cat tail grass blowing in the wind which is susceptible to artifacts when grading. The original file has no mud or artifacts at all. I apply some grading in CS5, convert and export the original 59.94 fps to 29.97 fps to AVI lossless, and run the avi file through avisynth with denoise and get a really good quality MP4 file with hardly any artifacts or mud. The grading holds up quite nice. However, when I upload the MP4 file to Vimeo, there are mud and artifacts visible that were not visible on the uploaded file. Hope I didn't miss something, but was a tad deflated when I saw the results. I thought I might have found the holy grail. :cheesy:
 
Great to hear you tried it out, Bobomatic!

Yes, while it is generally a good idea to encode "excessively" when uploading to a video hosting site, it comes to the point of diminishing returns because their encoding is just so simplistic. Youtube and Vimeo use their own branches of x264 actually, so I'm not sure how up-to-date they keep them, and on top of that, they intentionally tune them down for speed and filesize. I find Vimeo to hold up more than Youtube, but both will crush a lot of sources...

The price you pay for free!
 
I've been messing around with Stax and your guide definitely helped a lot! So thank you!

I have a question though, I'm trying to render out some low light HDSLR footage that has a bit of noise in it and I can't seem to find a happy medium that reduces noise but also doesn't create the pixel blocking artificialness of compressed video. I'm rendering out for Blu-Ray so I'm using that preset. I tried a CFR based render and bumped it all the way up (down) to 18 to get the highest quality and then rendered on placebo. Have any suggestions about getting a better render without all of the pixel blocking?
 
You must log in or register to reply here.
Back
Top