Best Codec to use for Lossless? [Archive] - DVXuser.com -- The online community for filmmaking

Zacatac

01-07-2009, 08:14 AM

Hey guys,

So, I have the HMC150, have the shot .mts files, and I have Voltaic to convert to a format easier to use (I'm on the last generation MacbookPro) I've already converted it once to Photo-Jpeg medium quality to see, and its looking good, but it does add a lot of pixelation, and the file size is somewhat large.... is their any other codec that gives the same, if not smaller file size, but less pixelation? or am i asking for something to much?

Thanks,
Zac

David Saraceno

01-07-2009, 11:21 AM

I use DVCProHD.

PerroneFord

01-07-2009, 11:38 AM

DVCProHD is HUGELY lossy. Why would you recommend that? It takes his 1920x1080 file and converts it to 1280x1080 right off the bat. Not to mention what it does inside each frame.

If you really want to squeeze the files down, you could use wavelet compression like Jpeg2000. If you want to stick to a full raster, small file size with good looks, try the Avid DNxHD codec, or Cineform. Both can be had in 8 to 10-bit. with the Cineform going all the way through RAW 4:4:4.

If you truly want lossless there are several included with the machines depending on your container. Both AVI and MOV containers have lossless options or uncompressed options but the file sizes are HUGE.

There is always high bitrate mpeg4 as well. At 40-50Mbps, it stands up VERY nicely and that's why we see so much of it on BluRay. I've also been incredibly impressed with the Windows Media Encoder when you give it that much bandwidth, and it's quite popular on some reference grade BluRay's also.

The problem you have is that the better a job the codecs do squeezing down to get smaller file sizes, the harder your computer has to work to unsqueeze them to play them back. So while Mpeg2 is reasonably squeezed, most modern computers can play them back without too much trouble. Compare that to Mpeg4/AVCHD and though it makes very clean and small files, it takes a TON of horsepower to get it to play back cleanly in edit.

I prefer to use DHxHD and Cineform, because they squeeze fairly decently, but the quality is excellent, and so is the playback.

kurtmo

01-07-2009, 08:45 PM

The mainconcept DVCProHD converter left my 1920x1080 files as 1920x1080. Yes, the files are bigger.

I know it's not really an option for Mac folk.

Edit: Oops, they are 960x720.

David Saraceno

01-08-2009, 11:09 AM

DVCProHD isn't hugely lossless and for us there is a trade off with editing ease.

Also, because we intercut with a 170, we stay with DVCProHD.

However, AVCHD starts off heavily compressed. You really improve nothing at all by going to a lossless codec in my view

PerroneFord

01-08-2009, 11:43 AM

If DVCProHD doesn't truncate the footage to 1280x1080 and leaves it at the standard resolution, then I see the value. It's a good 100Mbps codec. And if you are bringing in other DVCProHD sources, I can see where this would make sense.

But on it's own, I would not use that codec. There are better editing codecs out there. 100Mbps at full res is a pretty heavy burden for online editing. Especially on an older, 1 hard drive latptop.

pailes

01-08-2009, 02:14 PM

PerroneFord is right, DVCProHD is 1280x1080. If you're on a Mac with FCS I'd recommend ProRes 422, it performs very well.

You also need to realize that true lossless doesn't make sense with HD video. ProRes 422 is what Apple calls "visually losless" but it is in fact a lossy intraframe codec just as DVCProHD. You will just have a hard time in recognizing compression artifacts with ProRes.

PerroneFord

01-08-2009, 02:53 PM

While ProRes is awesome for the Mac, I am not recommending it anymore simply because it is Mac ONLY. In this world of collaboration, having ANY codec that cannot pass platform to platform is a non-starter. Or at least it should be.

Look a these forums for a week, and look at all the problems people have trying to get their video from one place to another. ProRes, Cineform, AIC, etc. I'm done with all of it. Jpeg2000, DNxHD, AJA/Blackmagic all eliminate these problems and can drop into FCP, Vegas, Avid, and Edius with no fuss. Also works 32 or 64 bit OS's and some are 10bit so they hold up for color and other work.

Ed Kishel

01-08-2009, 04:34 PM

pailes

01-09-2009, 12:15 AM

While ProRes is awesome for the Mac, I am not recommending it anymore simply because it is Mac ONLY. In this world of collaboration, having ANY codec that cannot pass platform to platform is a non-starter. Or at least it should be.

ProRes is available on Windows also. Apple released it a few months ago. ProRes is a really good codec and it has got quite some advantages if you're messing with it in FCP. As long as you don't excessively exchange the transcoded footage with the windows world, I don't see any reason to not use ProRes.

PerroneFord

01-09-2009, 05:50 AM

ProRes is available on Windows also. Apple released it a few months ago. ProRes is a really good codec and it has got quite some advantages if you're messing with it in FCP. As long as you don't excessively exchange the transcoded footage with the windows world, I don't see any reason to not use ProRes.

No, ProRes is NOT available for Windows. ProRes is able to be decoded in Quicktime player. Which is great if I want to WATCH it. I still can't edit it, or encode to it so that I could give a corresponding file back to an Apple user.

The DNxHD codec is every bit as good as ProRes and you can view and edit with it on both platforms for free.

PerroneFord

01-09-2009, 06:00 AM

I have been running the demo of Neo Scene by Cineform. The AVIs created from the AVCHD files are identical to my eyes. HOWEVER, the sizes do increase big time, 100mb AVC became 500 GB AVI :(

But much easier for NLE and slower machines to handle. Plus the Cineofrm AVI is 4.2.2 and 10 bit. Mac version arriving soon

http://cineform.com/products/NeoScene.htm

Havent decided if I will get it yet- vegas can handle AVCHD pretty well but with a Cineform AVI I can use PPro cs3

Cineform out of Vegas is not 10-bit. You can't do 10bit in AVI from Vegas which is why I switched to quicktime based files.

I am guessing your 100MB to 500 *GB* is a typo. I am also going to assume you are not familiar with uncompressed video editing. I'f you are new to this, or only familiar with DV then the file sizes will seem large. But for those of us who had to push around uncompressed SD files because we were doing color or other work, these HD sizes are AMAZING. Uncompressed 8-bit SD is 176Mbps. I am putting archival quality 1080/24p into the less space right now.

pailes

01-09-2009, 07:04 AM

No, ProRes is NOT available for Windows. ProRes is able to be decoded in Quicktime player. Which is great if I want to WATCH it. I still can't edit it, or encode to it so that I could give a corresponding file back to an Apple user.

Ok, I didn't know that, but then again, I don't exchange footage with the windows world and if I needed to I can still transcode the original footage into some different codec if necessary.

The DNxHD codec is every bit as good as ProRes and you can view and edit with it on both platforms for free.There are a lot of benefits when using ProRes 422 in Final Cut Pro, especially performance-wise. You need to prove that DNxHD can keep up with all those advantages before stating something like that ;)
Do you have detailed information on how fast decoding/encoding of DNxHD is in comparison to ProRes 422?

PerroneFord

01-09-2009, 07:14 AM

There are a lot of benefits when using ProRes 422 in Final Cut Pro, especially performance-wise. You need to prove that DNxHD can keep up with all those advantages before stating something like that ;)
Do you have detailed information on how fast decoding/encoding of DNxHD is in comparison to ProRes 422?

No, but Avid does since they built the codec to compete with ProRes. My guess, is that since they use very similar encoding methodology and very similar bit rates, is that DNxHD is going to be very similar in speed to ProRes. Since Avid needs to support both Windows and Mac, they needed something that could walk back and forth cleanly between the two platforms. So they essentially built a ProRes clone, and made it open standard (you can download the source code), and got it into SMPTE standard unlike ProRes.

The fact that I can watch 1080p 175 mbps encoded DNxHD on laptop in real time, tells me that it's going to be just fine in a MBP or similar.

[edit]

It's probably not correct to say that Avid built the codec to compete with ProRes. They had a need for a similar codec, that was platform independent. So they built one. It works very well. Looking at the client list for who is cutting with it in HD instead of SD, I'm impressed.

xmephestox

01-09-2009, 08:06 AM

avid also has a multitude of variations with the codec too for offline editing. pro res is 1 data rate, and 1 way of compression so you get the same file sizes across the board. DNxHD you with the different variations you can throttle ur bitrate down even more for quicker editing before you online, and it still looks pretty good even at the offline side of things.

i also don't think pro res will work on a mac without a certain version of FCP installed on it. i couldn't find any pro res quicktime codecs online at all, and i took a file i had as pro res to someone's mac for preview where they didn't have FCP installed and it didn't work. newest qt version and everything.

pailes

01-09-2009, 08:08 AM

@PerroneFord:

Lots of interesting information. I will definitely look into the DNxHD codec and run a few tests on my own. If it can keep up with ProRes 422 on my Macs this might be worth a switch.

PerroneFord

01-09-2009, 09:15 AM

@PerroneFord:

Lots of interesting information. I will definitely look into the DNxHD codec and run a few tests on my own. If it can keep up with ProRes 422 on my Macs this might be worth a switch.

I've posted a lot about this codec in recent days (weeks) and I probably sound like an Avid shill. But I don't even own Avid, I cut on Vegas.

I have a need to move files back and forth with PC and Mac users, and also with local broadcasters as they move from DVCam/DVCPro to HD flavors. DNxHD holds up everywhere and everyone can read it. I can even put copies of the Mac and PC codec on the deliverables to ensure the person getting it can read it, edit it, and render it back out if necessary. I WISH Apple had done something similar with ProRes (it's not THAT hard) but they chose to remain closed.

I was kinda on the fence about whether this codec would hold up until I really started looking into who was working with it and deeming it good enough. Check some of this:

Iron Man
http://digitalcontentproducer.com/mil/features/video_dnxhd_marvel/index.html

Focus Features
http://biz.yahoo.com/iw/080603/0403128.html

The Mummy
http://digitalcontentproducer.com/mil/features/mummy_reboot/index2.html

NBC
http://www.broadcastingcable.com/article/CA6583785.html?q=dnxhd

CBS
http://www.broadcastingcable.com/article/CA6576275.html?q=dnxhd

David Saraceno

01-09-2009, 10:13 AM

What would one suggest as the best virtually lossless cross platform FREE EDITING CODEC that would available for NLEs on both platforms?

Zacatac

01-09-2009, 02:26 PM

So i tried to export AVCHD mts files to AIC full quaility, but im still seeing alot of banding and digital noise...

is this something Im going to live with?

Thanks,
Zac

pailes

01-10-2009, 01:45 AM

So i tried to export AVCHD mts files to AIC full quaility, but im still seeing alot of banding and digital noise...

is this something Im going to live with?

Thanks,
Zac

If you read this thread carefully you might find other options than AIC :Drogar-Dum(DBG):

Zacatac

01-10-2009, 08:22 AM

If you read this thread carefully you might find other options than AIC :Drogar-Dum(DBG):

yes, I'm using voltaic and PPro Cs4, yes i have no option to convert to DVCproHD, and I'm not dishing out the money for cineform just yet, now you can call me a hypocrite after spending money on the cam, but is there any free codec, besides AIC, cause i can't find the correct outputs for Prores or DVCproHD

I'm also on a Mac so no PC based AVCHD to DVCproHD converter

pailes

01-10-2009, 09:31 AM

yes, I'm using voltaic and PPro Cs4, yes i have no option to convert to DVCproHD, and I'm not dishing out the money for cineform just yet, now you can call me a hypocrite after spending money on the cam, but is there any free codec, besides AIC, cause i can't find the correct outputs for Prores or DVCproHD

I'm also on a Mac so no PC based AVCHD to DVCproHD converter

If you're not using Final Cut Studio you won't have ProRes 422 as an option (and probably no DVCProHD either). These codecs come bundled with Final Cut Pro. If you need a free codec try the one suggested by PerroneFord (DNxHD from Avid).

Zacatac

01-10-2009, 11:18 AM

Thanks pailies for your help...

just one more question if you will... What do i use to convert .mts files AVCHD to DNxHD files?

Thanks
Zac

pailes

01-10-2009, 12:49 PM

I'm using Toast 9 currently. No problems so far.

Averdahl

01-10-2009, 01:22 PM

...but is there any free codec, besides AIC, cause i can't find the correct outputs for Prores or DVCproHD
Add the AVCHD files to Adobe Media Encoder and make sure to choose P2 Movie under Format in the Export Settings dialog. You can queue up all your clips and transcode them in one go.

Or, File > Export > Movie does the trick in PPro. Make sure to choose P2 Movie under Format in the Export Settings dialog.

The result is DVCPro HD in a P2 folder structure.

/Roger

xmephestox

01-10-2009, 08:01 PM

if you and DNxHD go to the avid websites and download and install all the codecs. then when you open up Adobe Media Encoder, the option is there for you.

adkimery

01-10-2009, 09:06 PM

No, but Avid does since they built the codec to compete with ProRes. My guess, is that since they use very similar encoding methodology and very similar bit rates, is that DNxHD is going to be very similar in speed to ProRes.
DNxHD is going to perform poorly inside FCP though because it's not an 'optimized' codec which means no real time performance. Every dissolve, filter, etc., that you add in FCP's timeline will have to be rendered. Cross platform, and cross program, sharing of files is just a b*tch right now.

-A

PerroneFord

01-11-2009, 05:58 AM

DNxHD is going to perform poorly inside FCP though because it's not an 'optimized' codec which means no real time performance. Every dissolve, filter, etc., that you add in FCP's timeline will have to be rendered. Cross platform, and cross program, sharing of files is just a b*tch right now.

-A

Fair point. So a Mac user could work natively in ProRes, and then Render DNxHD when the time came to hand off the files to non Mac people. Rendering the final in ProRes guarantees no one but Mac (FCP) people could use the "optimized" codec.

I think optimized means "works only with our product". I prefer to work with non-optimized codecs that I can hand off to anyone at any time.

Guess it depends on your needs.

Zacatac

01-11-2009, 08:39 AM

Alrighty, All figured out...

I'm a mac guy, but running adobe, so I've gone with DNxHD as my primary codec, and its working very nicely

PerroneFord

01-11-2009, 09:05 AM

Alrighty, All figured out...

I'm a mac guy, but running adobe, so I've gone with DNxHD as my primary codec, and its working very nicely

What bit rate are you using in the editor? And are you doing offline/online or just using one bit rate to edit and finish in?

matt s.

01-11-2009, 01:47 PM

PerroneFord

01-11-2009, 03:48 PM

Ive downloaded and installed the DNxHD codec as well and no problems transcoding with toast 9. I havent set up FCP yet to work with the files but now I'm confused at what i should transcode to as there are options i guess as far as bit rate is concerned. I'm guessing thats what the 90,60 etc is when you select it in the settings.

and whats up with the color after transcode it becomes really desaturated?

In general, the codec will offer a selection of three bitrates based on your source footage's size (1080, 720) and it's frame rate. Generally, it makes the most sense to select a bit rate that most closely matches the original source bit rate. For my Sony EX1 that records in 35 Mbps, I use DHxHD 36. Someone using the DVCProHD, would select something around the 115 Mbps.

In terms of color, it sounds like you have some RGB material that you've encoded into YUV color space. The DNxHD lets you choose either Rec709 or RGB color space in the options list. The PC interface seems to be screwed up though so it's hard to see certain things. This is not an issue on the Mac version.

Zacatac

01-11-2009, 05:26 PM

PerroneFord

01-11-2009, 06:41 PM

Drag both a 220 and a 145 rendered file on the timeline with the original on a separate track. Do a difference between them. Generally you can set it up as a compositing child. That will show you the difference. DNxHD 36 compared to my XDCam EX was essentially lossless. That is why I switched.

Edit:

Here are some full-sized screen grabs from a shoot I did a couple weeks ago. This frame grab is from the middle of a zoom-in with highly saturated colors. So nearly a worst case scenario.

Original File:
http://www.dvxuser6.com/uploaded/20610/1231879403.jpg

DNxHD 175x (10-bit):
http://www.dvxuser6.com/uploaded/20610/1231879485.jpg

DNxHD 36 (8-bit Proxy):
http://www.dvxuser6.com/uploaded/20610/1231879445.jpg

Difference between source and Proxy:
http://www.dvxuser6.com/uploaded/20610/1231879541.jpg

Difference between source and online version:
http://www.dvxuser6.com/uploaded/20610/1231879513.jpg

lirany

01-13-2009, 12:52 PM

Im debating still with 200 or 145 (i think thats it)

i haven't done that much testing yet, but with 8-bit 200 the only difference i can see why my naked eye between that and the original .mts, is its a bit lights, but curves brings it right back

Zacatac, how are you editing DNxHD in premiere? what preset do you pick so that you don't have to pre-render every clip you enter to premiere?

Thanks.

Zacatac

01-14-2009, 06:25 AM

Zacatac, how are you editing DNxHD in premiere? what preset do you pick so that you don't have to pre-render every clip you enter to premiere?

Thanks.

Im currently having to render everything that goes in, so thats a downside

ive recently just converted the original clip to every codec i could find... the best is Avid Packed Codec, as there is no difference in quality... the downside? 15 seconds in 1.8Gb...

next in line is Jpeg 2000 and Photo-Jpeg, which through my eyes is better than DNxhd

PerroneFord

01-14-2009, 07:48 AM

Well sure, an lossless codec is ALWAYS going to do better in terms of visual quality. Take Huffyuv, or any number of them. The problem (at least for me in Vegas) is that they aren't available in the 64 bit version, and on PCs, they typically can't offer 10-bit space.

Jpeg2000 is wavelet and I've played with it, but it's automatic in terms of bit rate. So if I am trying to lock to a bit rate that just won't do. PhotoJpeg and MotionJpeg are also reasonable, and I've used those to transfer files to Mac users in the past. Both can be lossless, but they are both *slow* and the files are quite large.

I've also used PNG to move stuff back and forth with Macs, and PNG is beautiful, but also slow and creates large files. There's ton's of codecs out there to do this work with, but for ease of rendering, control of bit rates, availability, and price, I still haven't found any combo better than DNxHD.

dr jones

01-16-2009, 01:20 PM

DVCPROHD converter is indeed lossy and that is why i don't use it anymore.

now i just use sony vegas 8 because it has full support for AVCHD. it edits the RAW AVCHD files like butter. no need to transcode first.

when shooting in 1280x720, the DVCPROHD will transcode it into 960x720. you will lose ALOT of your horizontal resolution lines.

i would rather edit in avid but until they release full support for AVCHD, i will edit in vegas so i don't have to convert by using the DVCPROHD transcoder.