AF100 and DNxHD.mov What it gives to my footage.

[Igelkott Film]

You do now the difference between RGB and video levels right? In my experience this was an issue when I worked in Avid but I haven't encounter these problems in PP and I only have two flavours these days and that's AVCHD or PRORES and they both are identical when it comes to levels. There's is no magic how the whole RGB vs video levels work. Video levels 16-235 were intended to have levels in a safe region with headroom (0-16 and 235-255). There's really no fitting to be made since video is RGB but what can happen is that when importing footage into Avid you have to make sure you use the appropriate video levels 0-255/15-235. So since DNxHD is an Avid format you have to choose the right video/RGB level when encoding. You can reverse what seems to be happening with footage from the Hyperdeck in DNxHD by choosing the corresponding level when importing footage. I can't do this since I have no DNxHD files to play with but I know from my experience from Avid that it sometimes caused wrong levels.

AVCHD is one effective codec but implemented in 8-bit you can see i.e banding when you start to stretch footage and remember you only have half chroma resolution in 4:2:0 so any manipulation is done with less resolution than you have in 4:2:2. You have an uncompressed recorder so experiment yourself what happens when you grade AVCHD vs uncompressed or DNxHD.

I'm learning about video levels right now. Fascinating stuff.

You do have DNxHD to play with because I posted some www.igelkottfilm.se/video/DNxHD_test_2.zip
And I work in Premiere cs5.5.2, I stated that to.

AVCHD is a bad ass codec, that's why I'm searching so hard for the visible difference.
Without visible difference there are no need for uncompressed.

 

[PDR]

AVCHD is a bad ass codec, that's why I'm searching so hard for the visible difference.
Without visible difference there are no need for uncompressed.

There is quite a visible difference , depending on the scene; have a look at some of the earlier threads when AF100 got released.

The more you stress the codec, the larger the difference. e.g. more motion, scene detail

The biggest differences are in gradient banding, chroma information, macroblocking - if you're only doing this for web, it might not be worth it, but when doing compositing, VFX work , maybe green screen, there are significant differences

 

[Alpha Beta Omega]

Uncompressed for the AF is just a wast of GB but 4:2:2 PRORES is not. I'll take a look at your DNxHD files.

 

[PDR]

There is quite a visible difference , depending on the scene; have a look at some of the earlier threads when AF100 got released.

The more you stress the codec, the larger the difference. e.g. more motion, scene detail

The biggest differences are in gradient banding, chroma information, macroblocking - if you're only doing this for web, it might not be worth it, but when doing compositing, VFX work , maybe green screen, there are significant differences

I should clarify, that the difference in banding isn't from 10-bit 0-1023 (it's only a 8-bit signal) , the difference in banding is from the higher bitrates that preserve more of the fine sensor noise, that sort of "hides" the banding as a dithering effect. The AVCHD compression tends to "smooth" everything over, so you get posterization along gradients

There were multiple threads with footage, examples, proof when AF-100 got released, I suggest you have a look

 

[Igelkott Film]

Now I now about video levels.

So I use the filter ProcAmp or fast color corrector in premiere and I can now adjust this.
This is good to know, would loved this information when I started with Premiere 7

 

[Igelkott Film]

Uncompressed for the AF is just a wast of GB but 4:2:2 PRORES is not. I'll take a look at your DNxHD files.

If you want you can have the tree scene to

 

[Alpha Beta Omega]

I pushed further just to see how much more detail actually was in that original AVCHD-file:

View attachment 49134
Ungraded original:
View attachment 49135

 

[Igelkott Film]

I'm all in on this now.

Does this difference hurt my footage in any way? When recording to DNxHD?


This summer I made a fishing DVD with clips looking like this

when it could of look like this

 

[Zachary Halberd]

There is only one way to conclude this issue and to quell any doubt that anyone may have. A proper camera test. The whole gauntlet. Do at latitude test. Print out a fine gray-shade gradient card on transparency and backlight it with a soft and perfectly even source, then shoot everything in various settings to see if there truly is a difference. Shoot a res chart, and a color chart.

I love that you are exploring your formats trying to eek the best out of your camera, but there is no control in these tests so people have no way of knowing how to legitimize your claim. It's just like publishing a scientific discovery in a journal, but not providing the industry standard testing procedures with it.

I for one, want to believe you! Keep up the good work.

 

[Igelkott Film]

There is only one way to conclude this issue and to quell any doubt that anyone may have. A proper camera test. The whole gauntlet. Do at latitude test. Print out a fine gray-shade gradient card on transparency and backlight it with a soft and perfectly even source, then shoot everything in various settings to see if there truly is a difference. Shoot a res chart, and a color chart.

I love that you are exploring your formats trying to eek the best out of your camera, but there is no control in these tests so people have no way of knowing how to legitimize your claim. It's just like publishing a scientific discovery in a journal, but not providing the industry standard testing procedures with it.

I for one, want to believe you! Keep up the good work.

Thanx.

I compensate by giving all the facts and raw files.

I really have to hunt to se the advantage
Except for workflow

 

[Igelkott Film]

Color Levels
For all non-Avid QuickTime codecs, these options allow you to specify whether the file being imported has pixel values that are stored using computer RGB values (where the 0-700 mV (video black to video white) voltages are mapped using the full range of 0-255, 8-bit) or ITU-R BT.601/ITU-R BT.709 values (where the 0-700 mV voltages are mapped using the 601/709 standard range of 16-235, 8-bit).


However, when importing movies compressed using any of the Avid QuickTime codecs, you should always set this option to 601/709, regardless of the color levels in the movie. This is because the Avid codecs internally encode the material using 601/709 color levels. You instead specify the QuickTime codec options in a third-party program whether the file you are creating or rendering contains a computer RGB or 601/709 mapping. Failure to do this with current versions of Avid editing applications results in a dramatically slow import and incorrect color levels in the imported element.

http://avid.custkb.com/avid/app/selfservice/search.jsp?DocId=392963#Color_Levels

One thing to keep in mind is that PPro uses Full Range RGB (i.e. when converting YUV422 to/from RGB444).

Also- when you import/export to/from DNxHD MOV files - they also have their own Color Level Setting embedded in the MOV file wrapper - either '601/709' Color Level (i.e .SMPTE RGB) or 'Computer RGB' (Full Range RGB) - and that embedded Color Level Setting in the DNxHD MOV file will determine whether the Avid QT Codecs decode/encode DNxHD to/from either SMPTE Range RGB or Full Range RGB.

http://www.dvinfo.net/forum/adobe-creative-suite/505301-mystery-pp5-5-darker-file-reading.html

DNxHD reports RGB values ( tho it intends 709)

http://forums.creativecow.net/thread/8/1028870

That's not a gamma shift issue, which by definition affects midtones.


The Luma graph from the AVCHD recording shows full range data (full swing), with values from 0-255, which is good for dynamic range but is illegal video (for broadcast purposes). It has both superblacks and superwhites, which is good for CC.


The DNxHD recorded by the Shuttle shows legal video ranges (studio swing 15-235). Looking at the luma density around the black point it appears the full range signal outputted by FS100 has been COMPRESSED to studio range and not clipped. It is arguably correct behavior, if one was to broadcast the recording directly.


The dynamic range can be stretched back in color correction as the information is not totally lost (it's a less than ideal situation though as the compression/stretching will induce artifacts).


I think there should be an option on how the data is to be recorded: full swing vs studio swing (Rec709).
Personally I would choose to record full swing (illegal video) at all times and color correct to legal range for broadcast delivery, as it keeps all options open (web, cinema).


The trick in AVID is to import full swing data as Rec709, which leaves it as is. If you specify RGB it will compress/rescale it upon import to Rec709 (studio range).

It looks like the Hyperdeck is exporting the recorded data with the wrong flag set in the header. The flag is telling your NLE to remap all input values to a reduced range. My workaround is to import into resolve, first, and export to OP1a. Alternatively, you can reset the levels after you import into your NLE.

http://forums.creativecow.net/thread/124/879630

 

[Igelkott Film]

Getting to know the enemy

Rec 709 and RGB

I did a search on HD monitors and color levels:

"As of 2007, sRGB is by far the most commonly used RGB color space, particularly in consumer grade digital cameras, HD video cameras, and computer monitors." link

"sRGB uses the ITU-R BT.709 primaries, the same as are used in studio monitors and HDTV,[1]" link

"ITU-R Recommendation BT.709, more commonly known by the abbreviations Rec. 709" link
______________________

What does DNxHD record in:

Video data block #1:
Frame Size = 1920 x 1080
Compressor = DNxHD 709
Quality = Most (5253,00)

AVCHD records in RGB
_______________________

I decided to compare how AVCHD and DNxHD looked like in the AVC-Intra 100 timeline in premiere pro
compared to the How it looks in the AF100, SmallHD, on a HDTV from a WDTV and on the computer.

AVCHD on the timeline

DNxHD on the timeline


On the:
- PC with the dynamic range set to full (0-255) in the Nvidia control panel, played with WMP both clips looks like they do on the timeline.
- PC with the dynamic range set to limited (16-235) in the Nvidia control panel, played with WMP the color level of the AVCHD gets matched to the DNxHD.
- HDTV from WDTV, the raw MTS file from the camera looks exactly like the DNxHD file.
- AF100 in camera, it looks exactly like the DNxHD file
- On the SmallHD it looks like the AVCHD footage as it is displayed n in the timeline.
______________________

How does RGB translate to Rec 709 .

Both use 0-255

But how I understand it is that in Rec 709, pitch black goes up to 16 and pure white starts at 235 but there is headroom in 0-16 and 235-255 where YUV data/color can be retrieved.

In RGB 0 is black and 255 is white.
______________________

Displaying and the image.

HDtvs and HD monitors use the Rec 709 to display pictures and the AVCHD is stored in RGB.
IF the video card don't compensate for that everything under 16 and over 235 is clipped.

-The camera makes the signal broadcast safe when sent to EVF and LCD.
-The WDTV makes the signal safe and as do the video player in your computer if you set it to limited.
______________________

DNxHD vs AVCHD

DNxHD records all the 0-255 and tells the software to read it as rec 709 in 16-235 space.
From what I have read its NOT an gamma shift and it does NOT reduce IQ

AVCHD is recorded and stored in the entire 0-255 space.
When you view the signal on a Rec 709 monitor (which is the biggest standard) the whites and blacks gets clipped.

And that's why the AVCHD looks doll on RGB monitors, its because you like the contrasty picture :beer:and now there are no clipped blacks and whites.. it looks like Rec 709
_______________________

The enemy..

Premiere Pro and other NLEs except AVID seems to only work in RGB space and not compensating for it on a Rec 709 monitor.
And what we see in that space only resonates in video players with full range, Youtube and Phones/tablets that also does not compensate for the Rec 709 display.

And that's not what we see in the camera, on our WDTV's, broadcasts and video players with limited range

The enemy seems to be that i don't see a choice in editing in Rec 709 or RGB and exporting in that manner.

When I'm playing the DNxHD on a limited range video player the picture stays the same as when i view on a full range video player
in contrast to AVCHD witch get clipped on a full range video player.
________________________

The problem..

Mixing Rec 709 read footage with RGB read sequence will cause truble.
________________________

Future

I think DNxHD and other codecs with similar behaviors works as the are intended and designed to work and don't expect them to change.
Its the NLE's thats going to have to adopt more tools for the difference in broadcast and web production.

 

[Shooter]

Having Nvidia video player settings to 16-235 and playing AVCHD and DNxHD in windows media player they looks the same.

The setting is making the AVCHD broadcast safe.

Are you sure? I dont see how the Player setting affects an export making it Broadcast safe.

 

[Igelkott Film]

Are you sure? I dont see how the Player setting affects an export making it Broadcast safe.

It only affects the video players who listen to it like WMP but not Premiere.
Your computer screen, if its not an RGB screen, needs an broadcast safe signal to view the entire range of the image.

That's what I have found out at least.

Its not hard to test and see if you have Nvidia, switch between the settings in Nvidia control panel while you play a AVCHD clip in WMP

 

[Shooter]

My experience of legalising video for Broadcast is that it is content dependent.

Both Chroma and video levels need to be "legalised" and it is not really a one-stop shop.

It is quite amazing how much you can drop chroma levels (in each color channel if need be) using various techniques including "soft clipping"...and it has no visual effect to the eye ...but on the scopes a different matter entirely.

Legalising luma is similar but easier to do.

 

[Igelkott Film]

How do I drop chrome levels in premiere pro? Saturation?

My experience of legalising video for Broadcast is that it is content dependent.

Both Chroma and video levels need to be "legalised" and it is not really a one-stop shop.

It is quite amazing how much you can drop chroma levels (in each color channel if need be) using various techniques including "soft clipping"...and it has no visual effect to the eye ...but on the scopes a different matter entirely.

Legalising luma is similar but easier to do.

I'm sure it is. But when I say broadcast safe I mean Rec 709 safe.
Try the Nvidia thing and tell me whats going on.

One big problem as I see it is the software in WDTV and Nvidia that squezes all of the signal within range, it brings forth things that was not intended to see in premiere unless you use a RGB monitor
because if you drop the levels so everything in the white is visible to Rec 709 and export it, Nvidia and WDTV wont know what you have done and your film will look flatt.

I wish a could specify in project setting what I'm working on so it would be properly read. And I know at least nvidia respects rec 709 flags.

 

[PDR]

AVCHD records in RGB

No, AF-100 AVCHD records in Y'CbCr (you can call it YUV if you want, but technically it's Y'CbCr)

On the:
- PC with the dynamic range set to full (0-255) in the Nvidia control panel, played with WMP both clips looks like they do on the timeline.
- PC with the dynamic range set to limited (16-235) in the Nvidia control panel, played with WMP the color level of the AVCHD gets matched to the DNxHD.

This is only small part of the picture and only pertains to you (other people might have different setup). How it displays depends on several other factors including decoder configuration, renderer configuration, monitor setup. Different players may react differently . When you use "overlay mixer" for a media player, then the graphics card overlay settings are used. There are 2 sets of controls for most graphics cards. One of them affects the overlay mixer, one affects other settings like desktop. The 2nd one would affect NLE's like premiere, but the 1st wouldn't - because NLE's don't use the overlay mixer. But if you setup your media player with a different renderer e.g. EVR, VMR9 it might give completely different results that what you are seeing. The point is, your observations do not necessarily apply to other people - there are dozens of different factors that can change the apperance

How does RGB translate to Rec 709 .

Both use 0-255

Think of Rec709 as a set of equations (a transfer function) for converting Y'CbCr to RGB or vice versa

In 8-bit values both RGB and Y'CbCr range from 0-255. In 10-bit values 0-1023

But how I understand it is that in Rec 709, pitch black goes up to 16 and pure white starts at 235 but there is headroom in 0-16 and 235-255 where YUV data/color can be retrieved.

Using a limited range Rec709 transfer function:
Y' 16 or "black" gets mapped to RGB 0,0,0 . Y' 235 or "white" gets mapped to RGB 255,255,255 (<16, >235 get clipped)
CbCr from 16-240 also get mapped (<16, >240 get clipped)

Using a fullrange Rec709 transfer function, Y'0-255 and CbCr 0-255 get "mapped" to RGB 0,0,0 - 255,255,255 . This is also known as "studio RGB" transfer function

HDtvs and HD monitors use the Rec 709 to display pictures and the AVCHD is stored in RGB.

AVCHD is stored in Y'CbCr, most devices, HDTV, use Rec709 transfer function to convert to RGB for display on monitors

IF the video card don't compensate for that everything under 16 and over 235 is clipped.

It's supposed to be clipped for display on a properly calibrated setup, the actual video isn't altered . ie. you're not supposed to see the superwhites and superdarks. The video card settings don't affect the underlying video - only the display of it

DNxHD vs AVCHD

DNxHD records all the 0-255 and tells the software to read it as rec 709 in 16-235 space.
From what I have read its NOT an gamma shift and it does NOT reduce IQ

As mentioned earlier, the range in the recording is the same if you work or examine the file in Y'CbCr. DNxHD is very tricky because the software you use can reduce quality. Some will clip, some will clamp, some will shift gamma. It's software dependent.

AVCHD is recorded and stored in the entire 0-255 space.
When you view the signal on a Rec 709 monitor (which is the biggest standard) the whites and blacks gets clipped.

Only the display of superbrights/superdarks are clipped. Y' <16, >235 or CbCr <16, >240 . Underlying video isn't altered

 

[Igelkott Film]

Thanx for the corrections clarifications :grin:


I tested what would happen if I uploaded a DNxHD clip to Youtube and they interpret the clip the way you guys want to have it interpreted

 

[Steve Kahn]

Naive question: wouldn't an advantage of the Hyperdeck be the ability to capture detail in the blacks (where codecs capture the least amount of information) while simultaneously protecting the highlihgts?

That is, has anyone tested uncompressed recording (or hit bit rate DNxHD) when raising the master ped while generally underexposing a bit? In that way it seems to me highlights would be more protected while information would still be retained in the blacks.

 

[Igelkott Film]

Naive question: wouldn't an advantage of the Hyperdeck be the ability to capture detail in the blacks (where codecs capture the least amount of information) while simultaneously protecting the highlihgts?

That is, has anyone tested uncompressed recording (or hit bit rate DNxHD) when raising the master ped while generally underexposing a bit? In that way it seems to me highlights would be more protected while information would still be retained in the blacks.

No but that's something to try