redcode- amazingly good?!

[glenn chan]

This is bothering me because the .tiffs on the CML site show that Redcode is very very transparency. For 99.99% of the image, the compression is visually lossless (even if you zoom in 200%). The same cannot be said about 4:2:2 Y'CbCr (with no other compression), Cineform (no offence), DV's 5:1 DCT compression (or DVCPRO HD, HDCAM).

This begs the questions:
-Am I (not) seeing things?
-How does Redcode achieve such results?

My methodology:
Superimpose the images in Photoshop.
Set composite mode to difference.
Use the levels adjustment layer, drag the right slider to the left.
Make a new layer
Hit alt crtl shift E to merge visible to new layer.
Use that layer to help guide you to where the artifacts are.

Disable the layer, just compare the uncompressed versus compressed versions.
Zoom in if necessary (although this doesn't necessarily count, since real world viewers can't zoom in).

To compare to other codecs, take the uncompressed TIFF and encode that with the other codecs. The comparison may be slightly unfair, since Redcode can optimize itself for Red images (RAW space; it also knows the sensor characteristics).
Nevertheless, the Redcode images look much better than some of the other compression schemes out there. 8:1~9:1 in other schemes may be roughly equivalent to Redcode's compression (by 8:1, I am referring to a 8-bit RGB original compared to the 8-bit RGB compressed version in these other codecs).

[MMR]

Red Will Surprise Us
I Can Feel It.
Can Be Good Or Bad News!:zombie_smiley:

[evinsky]

Welcome to Redcode, can I get you another glass of coolaid?

[glenn chan]

Ok, well it seems that JPEG (exported out of Photoshop) at 100% quality may arguably do a better job. I'm unsure about my methodology since it may be comparing apples to oranges.

Nonetheless, I'm still impressed. And it's virtually visually lossless anyways, so there's not much more to be gained.

[FatBird19]

Ok, well it seems that JPEG (exported out of Photoshop) at 100% quality may arguably do a better job. I'm unsure about my methodology since it may be comparing apples to oranges.

Nonetheless, I'm still impressed. And it's virtually visually lossless anyways, so there's not much more to be gained.

your jpeg is only 8 bit. :)

[Graeme_Nattress]

I'm sorry Glenn, should I reduce the quality of REDCODE so you can see the artifacts more easily :-) How does it work - it works like this:

Pixels get transformed into Pixies. The Pixies are magic. We don't feed the Pixies, and with this crash diet, they loose weight. Now they're smaller, they fit into a smaller space. Once we get the Pixies into the computer to decompress them, we feed them lots of nice heavy electrons, they gain weight rapidly, and we turn them back into pixels for display on screen.

[adaml]

Graeme, you may be forgetting that this is a public forum. Now your competitors will know your secret technique and be able to copy you.

[pywl]

Pixels get transformed into Pixies. The Pixies are magic. We don't feed the Pixies, and with this crash diet, they loose weight. Now they're smaller, they fit into a smaller space. Once we get the Pixies into the computer to decompress them, we feed them lots of nice heavy electrons, they gain weight rapidly, and we turn them back into pixels for display on screen.

Lol, I felt like I was reading a bed time story.

Once upon a time in a magical Pixie land, the evil Graeme captured and inslaved a bunch of lovely sweet little pixies. The evil Graeme then starved these little pixies till they shrunk to a teeny tiny size. The pixies cried and flew around franically looking for food. As the pixies grew smaller and smaller the evil Graeme knew that his plan was working.

"HA..HA...HA" the evil Graeme cried it wont be long now.
Then one day the evil Graeme released all the pixies into a magical box. The box was filled with magical glowing electrons. The pixies ate and ate till they could no longer move. So full of glee they started to glow multiple shades of Red, Green, and Blue. Thus ends our tale of the Evil Graeme and his not so evil plan to make better Pixelies.

[lpcvideo1]

The pixies ate and ate till they could no longer move. So full of glee they started to glow multiple shades of Red, Green, and Blue.

Kinda like the rods and cones in our eyes, no?

http://www.thecandybaron.com/pics/1057.jpg

[Jannard]

OK... 6pm. Time for everyone's medication.

Jim

[filmmaker1977]

you're delayed jim.. :D here was 2am november 23 :)

[Blaine]

I'm sorry Glenn, should I reduce the quality of REDCODE so you can see the artifacts more easily :-) How does it work - it works like this:

Pixels get transformed into Pixies. The Pixies are magic. We don't feed the Pixies, and with this crash diet, they loose weight. Now they're smaller, they fit into a smaller space. Once we get the Pixies into the computer to decompress them, we feed them lots of nice heavy electrons, they gain weight rapidly, and we turn them back into pixels for display on screen.
That is just too damned funny Graeme.

[Tom Marshall]

OK... 6pm. Time for everyone's medication.

Jim

Does Guinness count as medication? :beer:

[firewallet]

Does Guinness count as medication? :beer:
I hope so... I am cracking open one now:beer:

Cheers everyone and I hope all our American friends have a fat and happy Thanksgiving weekend.:)

[filmmaker1977]

hey andrew and we over here?

[firewallet]

hey andrew and we over here?

Hahaha... You guys have fat and happy weekends every weekend!!! You live in the one of the most beautiful parts of the world and I cannot wait and share in that experience.:)

I will say that absolutely all of us are thankful for this project, the Team, and all the opportunities that this will afford us in our creative endevours. We can all partake in these thoughts...

[Tom Marshall]

Oh yes... happy Thanksgiving! :beer:

[glenn chan]

OK... 6pm. Time for everyone's medication.

I do believe I need medication. Going to 4:2:2 "uncompressed" would degrade the quality of Red footage (both uncompressed footage and Redcode). The quality of Redcode seems to exceed that of 4:2:2 uncompressed (uncompressed in the sense that there is no DCT or wavelet compression; compression only via chroma subsampling).

Capturing "compressed" (Redcode) gives you better quality than "uncompressed" (4:2:2). I do believe medication is in order.

[Jarred Land]

its funny.. i remember when i first was comparing the two images (recoded and non redcoded) I spend about 10 minutes trying to figure out how i screwed up the encode... I couldnt visually see the difference as I expected. I had to actually do the Photoshop difference to make sure they actually where different.

Its amusing all the guys in the audience at Nuart that claimed they could spot the difference between the two in a second.. it took us 20 minutes staring 3 inches away from the 20 foot screen to spot even one artifact.

[glenn chan]

They probably should be subjected to a double blind test.

2- It might also be interesting to see if people can tell the difference when keying the footage.

[Nick_Shaw]

I did some tests pulling keys with Keylight in Shake from both the uncompressed and RedCoded images. Both gave fantastically good results, with the mattes being visually indestinguishable at 1:1.

With exactly the same parameters on Keylight, doing a difference between the mattes showed tiny differences, but nothing that would affect the result for a viewer.

Nick

[Graeme_Nattress]

I don't think people fully realize the danger that the YCbCr + chroma sub-sampling does to an image. It rips it apart from it's heart. Even with DCT based codecs you get better results staying 444 YCbCr and just compressing the chroma channels twice as much. Subsampling (as with interlace) is woefully inefficient compression.

Graeme

[taubkin]

Thank god we don't need it anymore... God bless the end of NTSC!

[JDHolloway]

I think somebody should write a wikipedia article on Graeme Nattress. :thumbup:

Any takers?

[billg]

"I don't think people fully realize the danger that the YCbCr + chroma sub-sampling does to an image. It rips it apart from it's heart....Subsampling (as with interlace) is woefully inefficient compression." Graeme

Does this mean that RED footage shot in progressive mode will fall apart when converted to the current interlaced NTSC SD broadcast standard, eliminating or reducing the benefits of downsampling from 4k and its original creamy goodness? If RED footage is destined for SD interlaced broadcast, will originating in 1080i be the best alternative?

[boothba]

Does this mean that RED footage shot in progressive mode will fall apart when converted to the current interlaced NTSC SD broadcast standard, eliminating or reducing the benefits of downsampling from 4k and its original creamy goodness? If RED footage is destined for SD interlaced broadcast, will originating in 1080i be the best alternative?

RED will look wondeful downrezed to NTSC (already tried it). Just make sure you use a high quality filer or high sample rate so you don't get zingy edges. And yes stay progressive and then cine-expand (add 3:2 pulldown) once you're in SD. Most SD tv content is shot progressive (on film @ 24p) and cine-expanded in telecine (this is sometimes done in the online, or through a standards converting box, but most of the time it happens in telecine during your one-light transfer or select scene transfer.) Most people would (and should) choose to stay for progressive for as long as possible.

[Bro Anansi]

If destined for broadcast, why not avoid the potential motion artifacts of pulldown and just shoot 30p?

Ian Sun
Anansi Moving Images
Red #1073

[billg]

30p is probably where I'm headed.

I just finished a documentary this past summer that was produced in 30p on a DVX100A and broadcast on public television. It was okay, but, of course, it looks better on DVD played on a progressive DVD player.

All of my experience is with video, and I'm not invested in making my stuff mimic the 24 fps cadence of film. I prefer the more immediate look of 30p and wonder whether, before long, it will be the dominant aesthetic in most genres--or the ones I'm interested in--anyway. As far as I'm concerned, I'd rather not be limited by the stu-stu-stuttery look of anything but the slowest of pans and action.

[glenn chan]

For distribution, chroma subsampling may not be a bad idea.
--At (very) low birates with DCT-based compression, you are better off having lower resolution (and more bitrate/pixel) than having high resolution. You see this effect in web streaming clips (even when you full screen them).
--For delivery for television, chroma subsampling works well for analog interfaces (which accept signals with chroma subsampling, such as component Y'PbPr and composite NTSC and composite PAL).

For digital cinema acquisition, Redcode demonstrates that chroma subsampling is a bad idea. Better quality AND lower bandwidth with Recode VS uncompressed 4:2:2.

For broadcast acquisition, uncompressed 4:2:2 may not be that bad an idea for workflow reasons. If the online system doesn't accept your data format (i.e. XDCAM is IT-based but most online systems won't take it), then your next best bet is SDI ingest.

2- With uncompressed 4:2:2, you don't lose a lot of quality. Even arguably with the really bad implementation of 4:2:2 in Apple's uncompressed codecs.

3- Interlacing on the other hand is something you should be much more concerned about. It's not good for acquisition for VFX work.
When you also want chroma subsampling, interlacing interacts with it. So with 4:2:0, you may get worse performance than 4:1:1. Chroma subsampling is less than half as efficient/good.
Many display devices will de-interlace the signal poorly, resulting in artifacts or resolution loss.

(Just my opinion.)

[glenn chan]

If destined for broadcast, why not avoid the potential motion artifacts of pulldown and just shoot 30p?

30p doesn't convert to PAL well. Ideally, you want to sell internationally (to make more money) so shooting 24p can be a better idea for that reason. If your material is unlikely to end up in PAL, then 30p may not be a bad idea.

[Bro Anansi]

Wasn't thinking about those PAL guys, being a little NA centric I guess.

[billg]

And what about post-2010 (at least in the USA) when the NTSC standard goes away (except for the gargantuan installed base of legacy TVs) to be replaced by either 720p or 1080i depending on the network? Where does that leave us?

[Graeme_Nattress]

Does this mean that RED footage shot in progressive mode will fall apart when converted to the current interlaced NTSC SD broadcast standard, eliminating or reducing the benefits of downsampling from 4k and its original creamy goodness? If RED footage is destined for SD interlaced broadcast, will originating in 1080i be the best alternative?

No. Why should it? If that were the case then 35mm film on SD would look bad, but it doesn't.

Graeme

[Graeme_Nattress]

And what about post-2010 (at least in the USA) when the NTSC standard goes away (except for the gargantuan installed base of legacy TVs) to be replaced by either 720p or 1080i depending on the network? Where does that leave us?

Shoot 4k as that gives you the best picture.

Also, everyone's forgetting about Europe, and the rest of the PAL world which certainly accounts for many many more viewers than NTSC-land.

I'd suggest 25p or 50p for acquisition or just stick with 24p. I'm sure some sensible US broadcaster will figure out how to send progressive 24p to homes without adding silly pulldown. Pulldown bloats the bandwidth for no benefit.

Graeme

[im.thatoneguy]

And what about post-2010 (at least in the USA) when the NTSC standard goes away (except for the gargantuan installed base of legacy TVs) to be replaced by either 720p or 1080i depending on the network? Where does that leave us?

That's a myth. There is no government mandate to end Standard Definition Television. Only analog transmission. People can still make and sell 480i tvs, but the broadcast signal needs to be a digital signal (DTV) which is most commonly 480p.

[TimurCivan]

red should liscence the codec to thirdparties, other cam manufacturers etc. not only would themy make a nice tidy profit, but its could become a new industry standard.

Whats the compression ratio btw? how small does it crush?

[Digigenic]

Whats the compression ratio btw? how small does it crush?
According to an interview with Ted Schilowitz in highdef magazine, Nov/Dec 06 - pg. 13
We have an advanced compression engine inside the camera that takes the 4K raw or 2K RGB on down and applies a wavelet compression that's visually at roughly 10 or 12:1. To give you an example, the raw 4K at 24 frames per second is 325 megabites a second...

[TimurCivan]

WOW.

Thats amazing. Damn you guys a re good.

[Gordon Prince]

If destined for broadcast, why not avoid the potential motion artifacts of pulldown and just shoot 30p?

30p doesn't convert to PAL well. Ideally, you want to sell internationally (to make more money) so shooting 24p can be a better idea for that reason. If your material is unlikely to end up in PAL, then 30p may not be a bad idea.If we are working in a fifty/fifty base (PAL vs NTSC regions), what's the best shooting frame rate for the best results?

24fps -> 25fps?

Or 25fps -> 24fps?

[Graeme_Nattress]

I like 25p because you can take it to 60i with a modified pulldown that keeps speed, pretty much. 24p won't go to PAL without a speed change. That, and more of the world is PAL than NTSC..... That's just my personal thought though.

Graeme