hyperdeck shuttle 2: DNxHD VS Uncompressed

D

diegocervo

Well-known member
Hi all,

recently I purchased a Shuttle for my F3 and today I did a quick test to see how the DNxHD codec performs.
To my surprise, I found out that the difference between compressed and uncompressed is not so small.

I uploaded a couple of tiffs to my dropbox if you want to download them.
http://dl.dropbox.com/u/2888648/DNxHDvsUNCOMP.zip

The uncompressed file shows smooth tonal gradations on mids and shadows while the DNxHD is blocky and banding is quite visible on the gray paper.
Converting from uncompressed to prores 422HQ shows no visual difference to my eyes, instead.

Screenshots come from Premiere with max render quality and bit depth set to on.

Hope this helps,
Diego

disclaimer: please forgive me for the subject in the images....I did a very quick and dirt test with 1 led light. I didn't attach any diffusion panel to the light, that's why there are striped shadows on the folders on left side.
 
Last edited:
It's 8 vs 10bit problem- quite obvious.

Premiere is not reading 10bit in DNxHD. To verify it load DNxHD to AE (should say trillion colors), set AE to 16 or 32bit mode and than compare.
Another free software, which can for sure read 10bit in DNxHD is Resolve Lite or you can convert DNxHD to v210 using ffmpeg/ffmbc keeping 10bit.
Be aware that 10bit support in NLEs is not that obvious and limited and you have to know/test that it works as it should. Uncompressed v210 (after DPX, TIFF) is the most universal format to pass 10bit data- has the best support in NLEs.

There should be not much difference between ProRes and DNxHD- 1st one is bit better, but it's small difference (not visible on 1:1 frame).

There is quite a lot confusion and wrong reports about DNxHD (and ProRes) being not good. It's not true- most cases are like yours and related to workflow problems (user error) not codecs being not good. In 95% cases there is no way to tell difference between uncompressed and DNxHD/ProRes (highest quality modes) on 1:1 frame size. Difference starts showing up when zoomed 2-4x, depending on the source nature.
 
Last edited:
thank you all for your suggestions.
@Richard. The screenshot comes from a sequence already set to max bit depth (in QT desktop preview mode).
@Kolak. DaVinci does an excellent work with .mxf files. I wasn't able to open dnxhd in .mov container though. Any suggestion?
I will give a look at ffmpeg also. I don't have AE, so I cannot test it.
Best,
Diego
 
I have no idea why Resolve won't read DNxHD in MOV- as far as I remember it was working fine with beta versions.

In this case (if you want to keep 10bit) you have to convert to v210 with ffmpeg/ffmbc- as I said- it's not that easy to have 10bit workflow.
 
Resolve's documentation lists the input file types that do work and DNxHD MOV files are not listed. I can't remember them working in any of the Windows betas either.

My experience under Win 7, is that AE CS5.5 also will not read the Shuttle's DNxHD MOV files, even though Premiere Pro CS5.5 will. The CS5.0 versions also would not read them and as a result, the PPro QT importer was fixed under CS5.5. You can't just swap "good" and the "bad" QT importers, as that will cause AE CS5.5 to crash. The reason for this problem is supposedly related to the 16 audio tracks in the Shuttle MOV files.
 
jabloomf1230 said:
Resolve's documentation lists the input file types that do work and DNxHD MOV files are not listed. I can't remember them working in any of the Windows betas either.

My experience under Win 7, is that AE CS5.5 also will not read the Shuttle's DNxHD MOV files, even though Premiere Pro CS5.5 will. The CS5.0 versions also would not read them and as a result, the PPro QT importer was fixed under CS5.5. You can't just swap "good" and the "bad" QT importers, as that will cause AE CS5.5 to crash. The reason for this problem is supposedly related to the 16 audio tracks in the Shuttle MOV files.
Click to expand...

It may be my imagination that one of the betas did support it, but I still have most of the installers, so can check.
Resolve documentation regarding codec support is outdated and not very accurate. SR codec files work on Windows, but no where mentioned in their doc.

If audio is the problem for AE than extract just video part and save it as ref QT (you just need QT PRO) or again- use ffmpeg/ffmbc with video copy and disabled audio command.
 
I played with DNxHD files from de Hyperdeck Shuttle vs uncompressed records and was disappointed. First, I have to say that there is a huge color difference between DNxHD and uncompressed. I think it is due to the color range of the record : 0-255 for uncompressed and 16-235 for DNxHD (see : http://forums.creativecow.net/thread/124/879630#879752). I imported DNxHD in AE after I removed audio and it worked. However switching from 8 to 16 and even 32 bit had change nothing to the issue. Mids and shadows still appears exactly the same : blocky and banding. For the moment I think DNxHD files from Hyperdeck are worst than onboard XDcam recording of the F3. By the way if someone's able to make DNxHD from hyperdeck looks great, then just tell us how to edit those files.
 
What version of AE do you have?
Check if it says Trillion of Colors for DNxHD file.

I'm 95% sure it's all user error :)
If it does not say trillion of colors than you have to add these lines to AE QT rules file:

<QTCodec codec='AVdn' vendor='****' platform='mactel' direction='encode' versionlow='0x00000' versionhigh='*' gammatag='false' cvbuffertag='0' deepdecodefourcc='b64a' decodefourcc='argb' />
<QTCodec codec='AVdn' vendor='****' platform='mactel' direction='decode' versionlow='0x00000' versionhigh='*' gammatag='false' cvbuffertag='0' deepdecodefourcc='b64a' decodefourcc='argb' />
<QTCodec codec='AVdn' vendor='****' platform='windows' direction='encode' versionlow='0x00000' versionhigh='*' gammatag='false' cvbuffertag='0' deepdecodefourcc='b64a' decodefourcc='argb' />
<QTCodec codec='AVdn' vendor='****' platform='windows' direction='decode' versionlow='0x00000' versionhigh='*' gammatag='false' cvbuffertag='0' deepdecodefourcc='b64a' decodefourcc='argb' />

It's xml file located in (on Windows):

C:\Users\xxx\AppData\Roaming\Adobe\Common\
XXX is the current user name- one which you are logged in. You need to turn on Show Hidden files to see this folder.

Open it with Word Pad/notepad (don't use Word) and add these lines- make sure you keep proper format.
After adding them open xml file in Internet Explorer- if you see proper xml structure than it's all fine- if not you have to make sure formatting is not changed. These lines are already there by default in AE CS5 and up if I'm correct.

After this AE will see 10bit in DNxHD files. Another thing- you may have to add Levels filter (set 16,235 in 8 bit AE mode) to bring proper black level. You can also do it by using proper working color space and source file interpretation.

I can also suggest converting to v210 using ffmpeg.

ffmpeg -i "d:\my_dnxhd.mov" -threads x -vcodec v210 -acodec copy "d:\my_v210.mov" (or .avi).

x= number of cores in your machine
Depending on .mov or .avi you can convert to AVI or MOV files- audio part gets simply copied. If you want to disable audio than use... -acodec an
You just need to type proper paths to your source and output file.

Again- there should be no visible difference between DNxHD (also ProRes) and uncompressed on 1:1 frame. You need at least 2x zoom to start seeing differences. 10bit workflows are still bit problematic and you have to test it and know what yo are doing- don't ever assume that if your file is 10bit than every software will read this full 10bit data- very wrong assumption.
 
Last edited:
Ok. Thanks Kolak. I work on AE CS5 and it runs on mac. When I import DNxHD files, AE recognize it and says trillions of colors.


Here are 4 pictures captured from AE at 400% :


1. A DNxHD file at 8bpc. AE don't read the 10 bits from the DNxHD and of course, compression artifacts are visible. Watch the horizontal stripes and flat blocks.
2. The same DNxHD file but at 16bpc. I think AE should read the 10 bit from DNxHD. But compression still shows same artifacts.
3. An Uncompressed file at 16bpc. It is more grainy. But in fact, I think DNxHD codec flattens the grain and makes blocks. It is especially obvious in mids and very high lights when the movie is running. It makes flat moving areas.
4. An XDcam file from the F3 at 16bcp. It is closer to uncompressed than DNxHD, but of course noisier.


Perhaps I missed something. But I still think DNxHD from hyperdeck is not so good.
 

Attachments

Last edited:
I think it's all fine. Quality order is as expected- v210, DNxHD, XDCAM. I agree that DNxHD looks bit flat.
There are still issues with your comparision, mainly related to blacks/gamma.

None of you grabs have the same black/gamma levels. V210 has some black levels, DNxHD is brighter, and XDCAM is way darker- this is wrong. They all should look about the same.
Turn off Color Management by setting Working Color space to none (right click on Project tab or click on color depth number without ALT).
Looks like v210 is read properly, than for DNxHD file you have to apply Levels Filter (change project to 8bit mode and set 16,235)- this should make it darker and looks as v210.
What is going on with XDCAM I'm not sure, but it's way darker than v210- something is not right here neither. Maybe it's gamma issue on MAC, but I don't use MAC so not sure how to fix it.
If you can upload 1sec samples of each file than I can have a look- use QT 7 PRO to trim files.
DNxHD at 4x zoom will show some differences to v210, but it should be way better than XDCAM. It's already on your grabs, just different colors make it bit not obvious. There is problem with flat areas and this can be some bug, but first we need to see it at proper black levels (your grabs are 8bit, so I can't simulate it). High contrast, but also flat areas are the first to get compressed, so these sort of shots will show some degradation, but XDCAM will never preserve them neither (specially at 8bit). DNxHD is not high-end format- it's made for broadcast to save space, not for film to achieve highest possible quality. If you want 2nd one use v210 or Cineform, which should perform better than DNxHD. ProRes is also bit better than DNxHD, but it also depends on implementation.
It's also quite easy to check if HyperDeck has some bug- export v210 in AE to DNxHD and compare :) There may be bug, as we have seen with other disck recorders- BM still runs on quite new firmware.
 
Last edited:
Kolak and Mathias, thanks for taking the time to test dnxhd files from the shuttle.

Just out of curiosity, do any of you know why recorders can encode to dnxhd (or whatever codec) in real time, whereas my mac pro 8 cores takes much longer for encoding from uncompressed to a NLE friendly codec? It takes around 1,5 second to encode 25 frames to proresHQ. Is it possible that this process is more accurate when done on a computer?

Best,
Diego
 
Not really- DNxHD encoding with proper software can be few x faster than RT- just a matter of good multithreading and Adobe products are not the best in the matter. It's down to codec implementation- most intermediate codec can achieve high encoding speed on multicore machines as they use only I frames.
Some well optimised codecs like Cineform or GV HQX can encode at 10x (or even more) faster than real time on good machine.
 
Last edited:
Here are 3 files.


1. DNxHD from the hyperdeck in .mov container.
http://dl.dropbox.com/u/64413757/DNxHD.mov


2. Uncompressed file from the Hyperdeck in .mov container.
http://dl.dropbox.com/u/64413757/Uncompressed.mov

3. XDcam from the onboard recording of the F3.
http://dl.dropbox.com/u/64413757/XDcam.mov



Files 1 and 3 were shot simultaneously.
File 2 was taken just 15 seconds after that first shot, so light may have changed a bit.


After Kolak's advice about 10bit vs 8bit I admit that DNxHD is far better then XDcam when it is properly read.
But one big problem for me is to deal with color shift. I was not able to make DNxHD looks well. I mean, if I try to export Uncompressed or Prores from DNxHD just to compare in QT or in PP, it looks awful…
 
Forget about comparison in QT player- this is not a tool for this- this what it displays is not what is stored in actual file data (to much processing is going inside) :)

DNxHD exported from AE (with levels filter to bring proper blacks back) to v210 should be fine- if it's not than you have some workflow issues again- probably QT gamma problems. I used it on Windows and everything was fine, but it's slow, so I advise to use ffmpeg.

I'm not a MAC user, so not sure what sort of problems AE or Premiere have on MAC. Told you- to get 10bit out of some codecs is still a mystery and to have full working 10bit workflow is yet another challenge :) I think I found quite nasty bug in Resolve (need to check it properly) also- so looks like the most reliable solutions are free ones- ffmeg/ffmbc - well, I'm not surprised at all :)
 
Last edited:
kolak said:
Not really- DNxHD encoding with proper software can be few x faster than RT- just a matter of good multithreading and Adobe products are not the best in the matter. It's down to codec implementation- most intermediate codec can achieve high encoding speed on multicore machines as they use only I frames.
Some well optimised codecs like Cineform or GV HQX can encode at 10x (or even more) faster than real time on good machine.
Click to expand...

Kolak, I tried compressor, mpeg streamclip and media encoder. Apart from compressor, which is even slower, the other two guys cannot convert uncompressed to prores in real time on a mac pro 2010 with 8 cores and HDDs in raid 0.
The same applies when I convert uncompressed to dnxhd.
Again, I don't believe a recorder in a tiny box has more power than my mac pro....where is the magic?
 
a general purpose processor with general purpose memory and a general purpose operating system with layers of buffers, instructions and security made to be stable and run multiple apps and an app inside an app compatible with different frameworks and legacies vs a dedicated device that works alone, without permissions, interruptions, sharing of space or memory or processing.

also some of these recorders are made with gpu and soc designs.
 
diegocervo said:
Kolak, I tried compressor, mpeg streamclip and media encoder. Apart from compressor, which is even slower, the other two guys cannot convert uncompressed to prores in real time on a mac pro 2010 with 8 cores and HDDs in raid 0.
The same applies when I convert uncompressed to dnxhd.
Again, I don't believe a recorder in a tiny box has more power than my mac pro....where is the magic?
Click to expand...

Magic is that none of the software, which you mentioned is well optimised :)

Carbon Coder encodes to ProRes at 5x faster than real time. FFmpeg can do 3x faster than RT.
How fast is your RAID? v210 is about 120MB/sec, so to be able to encode from it at eg 3x faster than RT you need fast RAID (4 drives) and ProRes has to go to another drive.
There is no magic-by rough rule all intermediate codecs can do realtime on 1 core (eg 2.5Ghz i7), so well done converter should do it much faster on 8 cores machine.
Try new FCPX- FCP 7 is has poor performance- I don't know how people can use it.
 
Did some tests today after realizing that banding was apparant in sky gradients...which tells me Premiere is seeing the DNxHD footage as 8bit. But I was using the stock "LEVELS" effect to correct the gamma irregularity...after removing this, *poof* the banding disappeared. So I replaced the stock LEVELS with the FAST COLOR CORRECTOR and no banding. As i read more about it, apparantly the stock effects that don't have the little lego-looking symbol with "32" inside it may not work right on 10-bit footage. Stick with the 32bit effects and you (should) be okay.

I also use Magic Bullet Colorista II and Looks II and they show the banding only when I'm working with the adjustment controls or even have the effect highlighted (selected) in the effects panel. As soon as I deselect the effect, the 8-bit blocky/banding issues go away.

I'm also working with the "Maximum Bit Depth" box checked in the sequence settings. This only allows for 10bit previewing, but doesnt effect the export. I also right-click the DNxHD file in the project panel and select "create new sequence from clip" which creates a AVCintra profile @60hz.

When I copy the grade to identical footage recorded simoultaneously in XDcam, the XDcam footage looks noticeably more blocky/bandy.

Not sure if any of my info helps...hope it does!! :)
 
Last edited:
You must log in or register to reply here.
Back
Top