HDMI Capture Problem SOLVED - AviSynth RULES!
- Thread starter Ralph B
- Start date
By normal fields, I mean ones that represent their own moment in time. Not a "blend" of the field before & after. It's more obvious if you follow the glow of the "lights" on the vehicle. On some fields, there isn't a "double halo" for the lights, it's only a "single halo". The "single halo" are "normal" fields
If every field were blended, you would end up with ghosting on every frame ; and that isn't the case here
When you examine any clip from any camera, any format . For example use
AssumeTFF()
SeparateFields()
and step through, in vdub or avsp, each field will move up & down slightly, and it's normal.
For example, your chroma test "24P UT 422.avi" will show this. When you weave() the fields back together you will get the original video back (so instead of looking at fields, you're looking at frames now)
If you want to look at Cb or Cr information in the fields, they "bob" too, it's normal
EDIT: the difference in CbCr "bob" appears to be from YV12 vs. YUY2 , but only for your UT video codec recordings . Your YV12 recordings don't "bob" when looking at CbCr information from the fields. If you use ConvertToYUY2() which upscales to 4:2:2, they begin to "bob"
For example, the native AVCHD clip is YV12, and it "bobs" when you use look at the CbCr information in the fields, which is expected.
Why are your UT 420 recordings acting this way? I have no idea
Anyways I don't think it's significant; the big problem here is the field blending in the chroma planes
But Ralph, if do any more tests, can you record in UT 422 ? It shouldn't make a difference for the ghosting but who knows....
Last edited:
I have only one thing to say:
NEVER NEVER NEVER NEVER NEVER NEVER NEVER NEVER NEVER NEVER record to a 4.2.0 CODEC!
ONLY RECORD TO 4.2.2
http://www.sendspace.com/file/2y02ss
This raw footage was recorded with UT 4.2.2.
The chroma fields behaves themselves and the script decodes it perfectly.
There's more good news. The metric on the duplicate frames is consistantly 0.0000, which means the threshold in fdecimate can be set low, where it won't delete good frames. I set it at 0.8 for my decode.
I'm sorry for the confusion. Recording to 4.2.0 really messed things up.
This evening I also recorded to Cineform and Blackmagic MJPEG, which are both 4.2.2 codecs. Both decoded fine.
Bottom line - the script works, as long as you record to 4.2.2.
NEVER NEVER NEVER NEVER NEVER NEVER NEVER NEVER NEVER NEVER record to a 4.2.0 CODEC!
ONLY RECORD TO 4.2.2
http://www.sendspace.com/file/2y02ss
This raw footage was recorded with UT 4.2.2.
The chroma fields behaves themselves and the script decodes it perfectly.
There's more good news. The metric on the duplicate frames is consistantly 0.0000, which means the threshold in fdecimate can be set low, where it won't delete good frames. I set it at 0.8 for my decode.
I'm sorry for the confusion. Recording to 4.2.0 really messed things up.
This evening I also recorded to Cineform and Blackmagic MJPEG, which are both 4.2.2 codecs. Both decoded fine.
Bottom line - the script works, as long as you record to 4.2.2.
Cosimo Bullo
Veteran
Who's going to write the easy(ish)-to-use white paper on this? And major THANK YOU in advance!!!
Read the first post of this thread. I am continually updating it. All the information is there to get you up and running.
Sure thing. I need to do some testing for consistency and look at the workflow and makesure there's not a difference in optimizing for an output YUY2 codec vs optimizing for the RGB stills like I have. I think I was finding that doing the FixBrokenChromaUpsampling() manually was shifting the chroma less or at least differently.. but I don't know if there's maybe something more process efficient with the actual filter vs manual. I'll simplify my code and get back to you.
Edit: Scratch that last part... don't see a shifting difference between the two ways of doing it anymore.. was prolly just a something I was doing at the time.
Last edited:
Chroma Code
Chroma Code
Okay, try this out for size...
The horizontal U subpixel shift is super subtle. Might not notice it at all, but it seems have a mild improvement on Barry's chroma charts. The vertical subpixel shift of the chroma and luma (meet each other half way) is far more distinct. It seems to be good as far as chroma and minimizes chroma combing/bleed even without shrinking down to YV12. Due to the subpixel shift in the luminance, you might notice a barely there difference in pixel sharpness, though I don't think it'd effect usable detail.
With just the chroma shift, there's seems to be less of a need to even do the manual squeeze down to 4:2:0 to smooth out the chroma further. On most images the difference might be neglegable, although again, on the chroma charts you can see an improvement. Try in with and without to see what I mean.
I forget if anyone has looked into this, but I was thinking of adding AVISynth code to adjust for the change in the colors on the HDMI, however the AVCHD colors may be altered by camera settings in ways the HDMI is not, which would make static code for that frivelous unless I make variable to coincide with every color/picture setting on the GH2. Anyone have any insight into this?
On the one piece of footage with the color chart and the loose change, the final HDMI with all the modifications, looks very similar to the AVCHD. Differences in the HDMI are a half-pixel horizontal shift, those color changes people have noted, and a subtle increase in detail/noise.
Still not entirely sure how significant the difference would be for chroma keying, although as people have pointed out the AVCHD does seems to reduce detail in shadows and a vast majority of the footage I've tested on is ideal footage... nothing that would stress the AVCHD motion/detail wise.
Let me know your thoughts.
Chroma Code
Okay, try this out for size...
Code:
#Place this code after AssumeFPS()
# Doing this manually wound up not to be too neccessary.. so far.
FixBrokenChromaUpsampling()
# Seperates the YUV channels. YV24 seems neccessary for the split and remerge to work.
ConvertToYV24(chromaresample="spline36")
Y = Tweak(sat=0, coring=false)
U = UToY()
V = VToY()
# Shifts the YUV channels individually. In this case subpixel(halfpixel) shifting.
U = U.Spline36Resize(u.width,u.height,0.5,-0.5)
V = V.Spline36Resize(v.width,v.height,0,-0.5)
Y = Y.Spline36Resize(y.width,y.height,0,0.5)
# Manually squeezes the chroma down to 4:2:0 and back. Prevents further shifting.
U = U.Spline36Resize(u.width,u.height/2)
V = V.Spline36Resize(v.width,V.height/2)
U = U.Spline36Resize(u.width,u.height*2)
V = V.Spline36Resize(v.width,v.height*2)
# Recombines individual YUV channels.
YToUV(U,V,Y)
# Depending on the source file you may need to convert from YV24 back to YUY2/YV12.The horizontal U subpixel shift is super subtle. Might not notice it at all, but it seems have a mild improvement on Barry's chroma charts. The vertical subpixel shift of the chroma and luma (meet each other half way) is far more distinct. It seems to be good as far as chroma and minimizes chroma combing/bleed even without shrinking down to YV12. Due to the subpixel shift in the luminance, you might notice a barely there difference in pixel sharpness, though I don't think it'd effect usable detail.
With just the chroma shift, there's seems to be less of a need to even do the manual squeeze down to 4:2:0 to smooth out the chroma further. On most images the difference might be neglegable, although again, on the chroma charts you can see an improvement. Try in with and without to see what I mean.
I forget if anyone has looked into this, but I was thinking of adding AVISynth code to adjust for the change in the colors on the HDMI, however the AVCHD colors may be altered by camera settings in ways the HDMI is not, which would make static code for that frivelous unless I make variable to coincide with every color/picture setting on the GH2. Anyone have any insight into this?
On the one piece of footage with the color chart and the loose change, the final HDMI with all the modifications, looks very similar to the AVCHD. Differences in the HDMI are a half-pixel horizontal shift, those color changes people have noted, and a subtle increase in detail/noise.
Still not entirely sure how significant the difference would be for chroma keying, although as people have pointed out the AVCHD does seems to reduce detail in shadows and a vast majority of the footage I've tested on is ideal footage... nothing that would stress the AVCHD motion/detail wise.
Let me know your thoughts.
GrgurMG - if you make an avsi for distribution, can you make provisions for 2.58 vs. 2.6 , or make a note of which avisynth version to use? It might shift the wrong way otherwise
With respect to color, the difference between Ralph's native AVCHD sample vs. HDMI 422 recording was ~80-85% saturation (the AVCHD was more saturated , so ~0.80-0.85 was required to make them roughly equal). It seemed to be similar on the "loose change" footage . I don't know if this relationship holds true on other recording setups. I don' t know what the camera settings were, but bringing down the saturation on the AVCHD (instead of bringing up the HDMI) seemed to make more sense, because the oversaturation obscured details (like the scratches on the pocket knife)
With respect to color, the difference between Ralph's native AVCHD sample vs. HDMI 422 recording was ~80-85% saturation (the AVCHD was more saturated , so ~0.80-0.85 was required to make them roughly equal). It seemed to be similar on the "loose change" footage . I don't know if this relationship holds true on other recording setups. I don' t know what the camera settings were, but bringing down the saturation on the AVCHD (instead of bringing up the HDMI) seemed to make more sense, because the oversaturation obscured details (like the scratches on the pocket knife)
Well, my current code I believe only works for AviSynth 2.60 and I'm use the Sept 27, 2009 build. I was having various issues with 2.58 and even an older 2.6 (i.e. just black video and no errors) although maybe later I'll try to jump back to 2.58 to test it out again. I'm just reading now about a know issue with 2.6 (or at least some version) where YToUV() doesn't work properly with YUY2... which is why my code wasn't working without ConvertToYV24()
YV24 (Y'CbCr 444) wasn't introduced until 2.6 , so that code would have to be modified for 2.58
Overall , that 2.6 build is fairly stable . There are a few bugs (and they have been submitted) but they shouldn't affect this workflow if it's just for IVTC and chroma optimization. I would just be a bit "nervous" to recommend an alpha build for general usage
Overall , that 2.6 build is fairly stable . There are a few bugs (and they have been submitted) but they shouldn't affect this workflow if it's just for IVTC and chroma optimization. I would just be a bit "nervous" to recommend an alpha build for general usage
Okay... I just tested with the newest regular 2.58 non-Alpha build (2010-01-27) from SourceForge. Since the YToUV function isn't broken in 2.58 you don't need to include that ConvertToYV24. I normally convertoRGB and use ImageWriter to take stills to compare.. but in 2.58 vs 2.6 I'm getting a small unwanted shift that's seems hard to correct for, though I believe it's simply due to not being able to add chromaresample="spline36" to the RGB conversion. Looking at actual rendered video to compare, I think your fine as long as you don't change the colorspace. Also, the known 2.58 chromashift issues in YUY2<->YV12 might be showing, so if your going to scale the chroma down to 4:2:0, use the manual method that keeps it YUY2.... I think it's also just might be inferior due to not being able to specify a chromasample type for ConvertToYV12().
So should I ever get around to making an avsi, I'll try to factor in the differences.
So should I ever get around to making an avsi, I'll try to factor in the differences.
Here's an uncompressed frame grab from an HDMI capture. It shows the exquisite image the GH2 is capable of producing.
http://www.sendspace.com/file/b8laqq
This was shot with the 20mm pancake lens.
If you want to see something amazing, take it into Photoshop and apply Unsharp Mask - Amount 500%, Radius 0.2.
EDIT: I forgot to mention - this was shot through a window.
http://www.sendspace.com/file/b8laqq
This was shot with the 20mm pancake lens.
If you want to see something amazing, take it into Photoshop and apply Unsharp Mask - Amount 500%, Radius 0.2.
EDIT: I forgot to mention - this was shot through a window.
Last edited:
Matthew P
Well-known member
Wow, that's amazing; it looks as if it's from a photo camera, not a video one!
AUDIO QUICK AND EASY
If your HDMI capture device has analog audio inputs, you can capture audio along with the picture. As I wrote previously, the GH2 introduces a delay to the picture when it sends it out the HDMI.
Taking GrgurMG's excellent suggestion, in Avisynth, I have successfuly added a delay to the audio so it syncs with the picture.
In the script, simply add the line:
delayaudio(.07)
That's it. You now have frame accurate sound.
I've added delayaudio(.07) to the script in the first post.
CAVEAT: The number .07 works with the Blackmagic Intensity board. Other capture devices may or may not require a different setting. So, be sure to test.
If your HDMI capture device has analog audio inputs, you can capture audio along with the picture. As I wrote previously, the GH2 introduces a delay to the picture when it sends it out the HDMI.
Taking GrgurMG's excellent suggestion, in Avisynth, I have successfuly added a delay to the audio so it syncs with the picture.
In the script, simply add the line:
delayaudio(.07)
That's it. You now have frame accurate sound.
I've added delayaudio(.07) to the script in the first post.
CAVEAT: The number .07 works with the Blackmagic Intensity board. Other capture devices may or may not require a different setting. So, be sure to test.
Last edited:
Hrm, what exactly is everyone's workflow for encoding the corrected video? Now that I'm looking at actual video instead of stills.. I find that at least while using TMPGEnc I have an issue rendering out to YUV codes... or at least that's what it appears like when importing to After Effects. It would seem that I'm getting reintroduced combing/issues when spitting out to YUV... having tried both UT YUV codecs and MPeg2. When spitting out to UT RGB and Uncompressed RGB the image is clean in After Effects.
Avisynth version 2.6 works pretty cleanly when going to RGB when you leave the input video as the YV24. Some versions of 2.5 however, when leaving the video as YUY2, have the same issue spitting out to the UT YUV codec... which I think I corrected with using ConvertToRGB() at the end of my script.
I was clocking my render speeds for the 8sec footage from Barry's chart.. I was excited to get some AvisynthMT working.... but then I realized that was with the bad YUV output issue. I've only rerun tests for clean output on two versions.... 2.6 (Sep 27 2009) and 2.5 MT Version 5, Mod Seraphy (July 12 2009). I've gotten it working with the MT version, but not while MTMode is activated. Here's the breakdown starting from fastest...
2.58 MT (MT off) - Uncompressed RGB /w ConvertToRGB(): 0:33 sec
2.6 - UT RGB (UT 1 Thread mode): 0:38 sec
2.6 - UT RGB (UT 4 Thread mode): 0:40 sec
2.6 - Uncompressed RGB: 0:44 sec
My attempts to get Avisynth multithreading to work with this build of 2.58 MT have pretty much failed. I was getting some pretty improved render times with Avisynth MT mode earlier with some other MT version... lost track of which... but I had like a 0:22 sec render... but again that was with bad YUV.
Avisynth version 2.6 works pretty cleanly when going to RGB when you leave the input video as the YV24. Some versions of 2.5 however, when leaving the video as YUY2, have the same issue spitting out to the UT YUV codec... which I think I corrected with using ConvertToRGB() at the end of my script.
I was clocking my render speeds for the 8sec footage from Barry's chart.. I was excited to get some AvisynthMT working.... but then I realized that was with the bad YUV output issue. I've only rerun tests for clean output on two versions.... 2.6 (Sep 27 2009) and 2.5 MT Version 5, Mod Seraphy (July 12 2009). I've gotten it working with the MT version, but not while MTMode is activated. Here's the breakdown starting from fastest...
2.58 MT (MT off) - Uncompressed RGB /w ConvertToRGB(): 0:33 sec
2.6 - UT RGB (UT 1 Thread mode): 0:38 sec
2.6 - UT RGB (UT 4 Thread mode): 0:40 sec
2.6 - Uncompressed RGB: 0:44 sec
My attempts to get Avisynth multithreading to work with this build of 2.58 MT have pretty much failed. I was getting some pretty improved render times with Avisynth MT mode earlier with some other MT version... lost track of which... but I had like a 0:22 sec render... but again that was with bad YUV.
Cosimo Bullo
Veteran
Thank you Ralph!
Cosimo Bullo
Veteran
The amount of tweakable detail in the shadows and highlights is amazing.