HDMI Capture Problem SOLVED - AviSynth RULES!

Raw Footage

Here are two raw GH2 HDMI clips, one captured with UT 422 lossless codec and the other captured with Blackmagic MJPEG codec.
I did this because they're both excellent codecs, they're free, and anybody doing HDMI capture should seriously check them out. And not just for capture... they're both viable options to use for your renders.

UT is available here:
http://www.free-codecs.com/download/Ut_Video_Codec_Suite.htm

Blackmagic MJPEG is here:
http://www.blackmagic-design.com/support/software/

Click on the Intensity download for your OS. On the next page, don't enter any information, just click Download Now. Save the file. Run it. This will install the drivers for the Intensity, even if you don't have an Intensity card present on your machine. But along with the drivers, comes all the Blackmagic codecs. You're free to use these codecs in all your video applications, no restrictions.

A side note: The first time I used the Blackmagic MJPEG codec, I thought it was defective because the colors were significantly altered. It turned out, the codec was fine, but my Nvidia graphics card needed to have Dynamic Range set to Full (0-255).

And now the clips:

Dog Walking - UT 422.avi
http://www.sendspace.com/file/8o0mx1

Man on Roof - Blackmagic MJPEG.avi
http://www.sendspace.com/file/ww564y

Keep in mind that Man on Roof was shot through a dirty window. Dog Walking was shot through a clean window.
 
Ralph B said:
We need to stop right here. Are you shooting 24P or 50i into the Ninja? If you're shooting 50i, then you need do no further processing, and can go staight to editing (if my understanding of the PAL GH2 is correct).

If you're shooting 24P, then you need to go the Avisynth route.

I have neither a PAL GH2 or a Mac, so I can't help you much beyond this.
Click to expand...

Thanks Ralph: I'm shooting 24p, so, given that the HDMI out is 50i when recording 24p, I have to go AVISYNTH route...
 
XDCam great results on Mac

XDCam great results on Mac

As a Mac user i was not entirely satisfied with the results i got from the DNxHD codec. There were some gamma issues, with different curves in different color channels, and the chroma was a bit too high. Al lot of difference with the original HDMI recorded signal.

I've found a perfect solution to get 100% identical footage (pre vs post Avisynth processing) using the XDCamHD codec. A delight for pixel peepers! The bitrate is 92mb (higher result in buffer overflow errors, but 92 is already very high for XDCam). It gives much smaller files than DNxHD too.

This works with ffmpeg using the following commandline:

ffmpeg -i <input_file> -pix_fmt yuv422p -vcodec mpeg2video -flags +ildct+ilme -top 1 -dc 10 -flags2 +ivlc -qmin 1 -lmin 1*QP2LAMBDA -vtag xd5c -rc_max_vbv_use 1 -rc_min_vbv_use 1 -b 92000k -minrate 92000k -maxrate 92000k -bufsize 72816666 -bf 2 -aspect 16:9 -an -f mov output.mov

In the batch script it now looks like:


for %%a in (*.avs) do ffmpeg -i %%a -pix_fmt yuv422p -vcodec mpeg2video -flags +ildct+ilme -top 1 -dc 10 -flags2 +ivlc -qmin 1 -lmin 1*QP2LAMBDA -vtag xd5c -rc_max_vbv_use 1 -rc_min_vbv_use 1 -b 92000k -minrate 92000k -maxrate 92000k -bufsize 72816666 -bf 2 -aspect 16:9 -an -f mov %%~na.mov

You need to have the XDCam codec installed on the Mac only, AviSynth/ffMpeg does not need it. The XDCamHD codecs are automatically installed with the following "XDCam Transfer for Mac" application from sony (direct link): http://www.sonybiz.net/res/attachment/file/12/1166605189212.ZIP

Remark: on the Mac, the codec is reported and labelled XDCamHD/50mb but the bitrate is really around 92mb.


Cheers!

Martin Beek.
 
@martin - your probably don't want to include ildct and ilme flags for ffmpeg , as they are meant for interlaced footage - interlaced DCT scan and interlaced motion estimation
 
PDR said:
@martin - your probably don't want to include ildct and ilme flags for ffmpeg , as they are meant for interlaced footage - interlaced DCT scan and interlaced motion estimation
Click to expand...

Why do you always know freaking EVERYTHING, PDR?! It's SO 2010 to always know everything... ;-)

But.., a great observation as ever; if you'd be of the female persuasion (nothing reveals the opposite btw), i'd give you a big sloppy kiss.

As a reward i'll disclose my source. That means that i'm admitting that i've just googled it up...
The original commandline from the post below didn't work (first because of a pair of quotes in the wrong place, then because of some weird error).

http://www.itbroadcastanddigitalcinema.com/ffmpeg_howto.html

Maybe there is more that can be tweaked.

Salute,

Martin
 
In my opinion, MPEG2 isn't very good quality wise, even at 100Mb/s for longGOP . You need >250+ Mb/s I-frame only for typical content . In addition, FFMPEG's mpeg2 encoder isn't very good, even the highest quality unconstrained -qscale 1 -intra 1 -qmin 1, won't get that level

Even without pixel peeping, looking at 1:1 , you will see differences in quality. Fine details will be missing. I can show comparisions if you want. I thought the whole point of using the ninja (or any external recorder) was to retain those details.

Unfortunately the mac software handles many Y'CbCr formats inappropriately with incorrect gamma, and you cannot encode to prores on a pc :(

I would check again about your gamma settings, because many articles say it's fixed in newest releases of snow leopard, QT X. Apparently you can choose 2.2 gamma so everything looks the same on PC and Mac now (and the rest of the world)
 
Last edited:
Martin, here's some suggestions ... perhaps it's worth investing in Cineform - it's cross platform and has the requisite quality.

Another possiblity is to explore rendering the Avisynth output to the various Quicktime codecs that are available on the PC. For example, "Animation" is lossless at 100, and the quality is still quite good as you drop to lower numbers. Photo-JPEG at 100 is also worth checking out.

And here's a long shot - instead of using QTinput, use directshowsource, and then go through the DNxHD procedure. Just try it.
 
Animation is lossless at 100%, but only as RGB

A word of caution on using RGB intermediates ; if you don't convert to RGB using full range in avisynth (e.g. if let ffmpeg or quicktime do the conversion) you will clip your superbrights and superdarks . This is what Mac software is doing when it "sees" Y'CbCr formats like DNxHD as RGB (this is in addition to the gamma shift) . Your software needs to be able to "see" it as a Y'CbCr format, or you need to map full range if you to preserve all the data

I've mentioned numerous times the problems with directshow, i'll try not to belabor the point. (It's even more important now not to use directshow, because mvtools is being used - the forward and backwards vectors can get messed up ). If you decide to use directshow then make sure the output pin is YUY2 not RGB, or you will clip the data . Use info() to check. You can use graphstudio to map your directshow filters
 
LeavingTheCandy said:
Wow! What some Voodoo. I was thinking about trying my GH2 to my KiPro, but I think I'll have a beer instead.
Click to expand...

I hope you're not being scared off by all this tech talk. The basic process works extremely well, and is described in detail in the first post of this thread.
While there is certainly a learning curve, once you get over that hump, the process is completely automatic.

If you already have a KiPro, it would be a shame not to use it with the GH2.
 
Hey guys.

That's pretty disappointing (re. the XDCamHD). I've been accused of pixel peeping many times and this time i see no difference between the proresHQ and the Mpeg, but - OK- i trust your judgement and knowledge on the matter.

I use the latest setup and snow leopard, et cetera. Still, the DNxHD footage doesn't look even close to the proresHQ Ninja-recorded footage.

Two questions.

1)
Can you give us Mac users all some last and ultimate advise, apart from going the Cineform route, that would provide the best picture quality and replication of the original Ninja footage with FInal Cut Pro?

2) WHICH version of Cineform - if i want to follow that route - should i obtain and do i need to pay for both a PC license AND a Mac license?

Feeling the workflow is close to perfection, but the results still looks sh*t on the mac...


Thanks, Martin.
 
Last edited:
martinbeek said:
(re. the XDCamHD). I've been accused of pixel peeping many times and this time i see no difference between the proresHQ and the Mpeg
Click to expand...

Are you saying that rendering to XDCam and then viewing that file on the Mac, looks identical to the original Prores on the PC? Then use it! You've solved the problem!
 
Ralph B said:
Are you saying that rendering to XDCam and then viewing that file on the Mac, looks identical to the original Prores on the PC? Then use it! You've solved the problem!
Click to expand...
Well... what i'm saying is, that I can not see the difference, but then again, i have no calibrated monitor or special measurement equipment to judge the results. I am just confused, just as LeavingTheCandy, by all the tech details regarding RGB.
If you say that this is all but lossless, i'l take your word for it - no mistake - and i'll dump XDCam right away.

I will now - as a test - first follow the route mentioned in this article:
That suggests converting the DNxHD footage to Prores using Compressor; it seems to do a great job in restoring levels and gamma.
Let you know what i find. If you have suggestions, please let me know.


Martin.
 
martinbeek said:
Well... what i'm saying is, that I can not see the difference, but then again, i have no calibrated monitor or special measurement equipment to judge the results.
Click to expand...


You don't need any special equipment. Just take a screenshot of the same frame using the same method , and compare in photoshop or a browser

All that matters is if YOU can see the difference. If you think it's fine , then go ahead and use it ... But I'm telling you - it's not as good as DNxHD, not at those bitrates you're using. It might be "good enough" for whatever you're using it for...

If you use long GOP MPEG2, you will have fluctuations in frame quality. B-frames will be a lot lower quality. I-frame will provide more consistent interframe quality but you need about 250+ MB/s for similar results as ProRes422 or DNxHD175 .

This was taken from your examples:
http://www.mediafire.com/?135ece6dz1esbdg

Even without zooming you can see the differences. DNxHD does a fairly good job, but the fine detail and grain is gone in the MPEG2 encode, and dark shadow areas are negatively affected. The MPEG2 encode used even higher settings than the settings you were using.

Remember this was a fairly static scene. If you had more motion or complex content, it would have fallen apart even more. The compression simply isn't good enough at those bitrates. It will certainly be better than the onboard AVCHD @ 24Mb/s, but it's clearly worse than DNxHD in terms of compression


I am just confused, just as LeavingTheCandy, by all the tech details regarding RGB.
If you say that this is all but lossless, i'l take your word for it - no mistake - and i'll dump XDCam right away.
Click to expand...

The take home message about RGB is this: You need a format that Mac software will recognize as Y'CbCr , not RGB, otherwise it will clip data and screw up the gamma.

I suspect DNxHD is probably being clipped by your Mac software as well because it's treated as RGB - I would do some tests with full range (slightly overexposed) subjects

MPEG2 will work, but you need higher bitrates; 100MB/s isn't enough. I can't find any free encoders that will do this for you (I tried HCEnc, but it's not working properly with 422 or it's crashing on very high bitrates)
 
The one that works here for quicktime on the PC is uncompressed 2VUY . (When you re-open the exported file in QT it looks the same as the Prores). Theoretically it should work when transferred to Mac as well. Other forms of uncompressed don't seem get treated as uncompressed Y'CbCr, and other compression formats are mishandled even when encoded through quicktime itself.

Filesize will be about 4-5x larger than the ninja prores, but at least you won't be clipping data, or incurring compression losses. You can convert that to prores on the mac if you want to.

Add this to the end of the script, and then play the avs in vdub or a media player like MPC to write the file

ConvertToYUY2()
QTOutput("out.mov", format="2Vuy", raw="uyvy")
 
STOP PRESS!!!

http://www.matrox.com/video/en/support/downloads/

I just installed the Matrox VFW Software Codecs, and their MPEG2 I-Frame Codec is fabulous at high data rates.
And they have a version of it that runs on the Mac. We may just have found the ideal cross platform codec.

I ran it through five generations and it held up so well, that I recommend it without any hestitation.

The only question I have is, since it renders to an AVI file on the PC, will that be directly readable on a Mac?
 
Good idea Ralph!

You should be able to re-wrap into mov with ffmpeg

ffmpeg -i input.avi -vcodec copy output.mov

You can batch re-wrap with a .bat file as well

Whether or not FCP "likes" it is another matter... IIRC, some MAC nanoflash users had some problems with high bitrate I-frame MPEG2 files

or does the MAC version allow you to parse AVI wrapped MPEG2 ?



EDIT: hmm maybe not so good. It forces a RGB conversion, and the decoder output is RGB if you are using matrox decoder...If you use other decoder you can decode in Y'CbCr, but the levels are clipped. I can't find a way to make it pass through Y'CbCr levels untouched, preserving all the data.
 
Last edited:
That setting should work for full range RGB to prevent clipping, but it's output pin is still RGB if you use the matrox decoder

Ideally you want Y'CbCr all the way through to avoid Y'CbCr<=>RGB conversions (slight preventable quality loss incurred)

Full range RGB won't look like the prores capture, it will look flatter, less contrast
 
Before we draw any conclusions, we need somebody with a Mac to test it and see how the files come across.

In any event, this is a helluva good codec, and I'm glad I found it. I'm going to use it in my own work.
 
You must log in or register to reply here.
Back
Top