ProRes RAW Getting Significant 3rd Party Support

Tamerlin said:
Do you have any actual proof of that? People keep making that claim, but haven't backed it up yet with anything but hearsay and the Atomos deal.



Yet makes its SDK available to anyone who wants to integrate it into post production software...



That's not the only reason for the encryption.



Been living under a rock?





Because of Apple's history... it should be pretty obvious.
Click to expand...

I misspoke. If Red had sued them it would be a matter of public record. More likely they threatened to sue BMD much as they did Atomos before Atomos caved to their demands. Red did sue Sony for the compressed RAW in the F5 and F55 and that case was settled out of court. But you are right, it could be some other company.

I just don't get how you think a company that encrypts video files so the owners of the camera can't read them is more open than a company (Apple) that licenses their format to more than 100 products. Red doesn't license their format to any other camera vendor, and sues to keep other camera makers from delivering compressed RAW.

We're going to have to disagree here.
 
Gary T said:
Red is the company that sued BMD and forced them to drop compressed Cinema DNG. Red is the company that forced Atomos to pay a royalty to record ProRes RAW. As far as I know doesn't license REDCODE RAW to any other recorder or camera. Red is so proprietary that their codec is encrypted so even owners can't freely read the file.
Click to expand...

For what it's worth the truth be known as much as Red likes to claim it invented a new codec in Redcode some years back there is fairly strong evidence that it is not so propriety and is in fact a derivative based off the the original JP2K 2001 released codec with added encryption. Early Redcode could be decoded and be displayed by a number of JP2K decoders... until it was encrypted. Early on in the piece when Redcode came out someone from Red mentioned and discussed the JPEG board in the camera and was quickly closed down. I have heard one knowledgeable tech say that on close inspection the ASIC used in the RED cameras looks to be an available off the shelf JPEG 2000 chip. Red very cagily refer to their Redcode in some of their online information thus: "REDCODE achieves this by using an efficient, variable bitrate "wavelet" technique. This is an extension of how JPEG2000* is able to achieve higher quality stills than other formats at the same file size." They then qualify this "*" by saying: "*Although JPEG2000 is a helpful analogy, RED®'s approach has been custom-designed for the needs of high data rate motion capture, and thus cannot be compared directly. Specific implementations of JPEG2000 also vary." Cannot be compared directly. A nice way of saying "No it's not JPEG 2000!" It may indeed may not be the original JPEG 2000 but the fact that they have shied away from any hard questioning as to who actually developed their proprietary codec has always left me wondering:

https://www.red.com/red-101/redcode-file-format

intoPIX one of the world leaders in cinema JPEG 2000

https://www.intopix.com/vision-and-cameras

even list Redcode as JPEG 2000 back in 2007:

https://www.intopix.com/Ressources/WPs_and_Sc_Pub/intoPIX - Pocket book about JPEG 2000.pdf

Also the fact that Red got very defensive back in 2013 and tried to sue both Sony and ARRI for infringing on "their Redcode" and ended up having to back off leaves me curious. Sony actually counter sued over that.

https://cinescopophilia.com/red-takes-civil-action-against-sony-over-camera-patents/

Seems Jannard will sue anyone who queries Redcode not being proprietary.

Chris Young
 
  • Like
Reactions: jcs
cyvideo said:
Also the fact that Red got very defensive back in 2013 and tried to sue both Sony and ARRI for infringing on "their Redcode" and ended up having to back off leaves me curious. Sony actually counter sued over that.
Chris Young
Click to expand...

This will probably drag this thread even more off topic - but I thought I remember reading that the Red / Sony legal threats back then were as much or more about compressed raw / log than a codec, per se... or maybe that's one in the same in this scenario?
 
Sony 16 bit raw is lightly compressed 3:1, even a dedicated pixel peeper would need luck to spot a difference between wavelet and dct at that level. Neither is it a serious strain on resources. The bit rate is about 1/3 more than standard E-OCN, about 2 1/2 times 10 bit XAVC 480, about 17% less than 12 bit Prores 4444, about 25% more than 10 bit Prores 422 HQ.

Highly compressed formats save recording space but place greater demands on playback during decompression. Sony AXS-R7 recorder and media are expensive but it's caused the prior AXS R5 recorder and media to plummet.

On a tight budget could be very hard to beat an FS700R w/ R5 recorder, memory card and reader for $3k.
 
Rumours were always that REDCODE was licensed / a version of Cineform.

My reading of their patent claim when they went after Sony was they claim compressed RAW and the way they handle the YUV RGB transforms (different compression schemes for the Y and UV)

JB
 
Tom Roper said:
Sony 16 bit raw is lightly compressed 3:1, even a dedicated pixel peeper would need luck to spot a difference between wavelet and dct at that level. Neither is it a serious strain on resources. The bit rate is about 1/3 more than standard E-OCN, about 2 1/2 times 10 bit XAVC 480, about 17% less than 12 bit Prores 4444, about 25% more than 10 bit Prores 422 HQ.

Highly compressed formats save recording space but place greater demands on playback during decompression. Sony AXS-R7 recorder and media are expensive but it's caused the prior AXS R5 recorder and media to plummet.

On a tight budget could be very hard to beat an FS700R w/ R5 recorder, memory card and reader for $3k.
Click to expand...

I've seen a few of those R5's on Ebay. What kinds of issues would you look out for if you worried buying used?

I've said it before, but I think Sony missed a huge opportunity by not letting Convergent Design and Atomos directly record the 12 bit linear RAW signal from their FS line. There's something to be said for being able to capture the original RAW signal instead of a one that has been recompressed, and if they had made it available I wouldn't have any interest in any other format.
 
Note that PP CC and Resolve support JPEG2000 (in addition to Cineform). The only reason I don't use it is it doesn't appear to have GPU/hardware acceleration (it's slow). Mathematically wavelets (DWT (discrete wavelet transform)) is a superior method for intraframe encoding since the compression error (artifacts) is spread out over the entire image (looking softer- less high frequency information) vs. DCT which produces significant localized (highly visible) artifacts, especially along macroblock edges. At higher bitrates, DCT looks fine, however DWT is more efficient, thus it's a superior solution, and why it's used for DCP in cinema.

If there are patent issues which prevent using the DWT in cameras, and especially with raw, the prior art shows that this should not be the case. Cineform and JPEG2000 preceded the patents, and there's no novelty of putting these codecs in a camera worthy of a patent: it's trivial and obvious. If I read the correct patent, the only element that appeared to be a trigger point in the claims was 2K or greater resolution. At that time I believe Cineform was "HD" resolution (1920x1080), and 2K > HD and thus "different". However a trivial bump in resolution is clearly not novel, especially to one versed in the art.

Years ago I was an expert witness in a patent lawsuit. I was surprised that something I'd been doing for years, that I found to be trivial, easy, and obvious, someone had applied for and was granted a patent. On the other side, I invented something truly novel, and ultimately had to abandon patent prosecution as the PTO would not allow any useful claims for clearly BS reasons of unrelated art. Later I wrote patents (while working for others that I did as part of the work), which weren't nearly as novel, and they were granted. So the system as it currently exists is extremely broken (and corrupt), and needs to basically go away. We need to do a common sense analysis as to why we need to protect ideas and the concept of power and value to society.

It looks like the iPhone now supports raw Bayer frame capture. Cineform is open source (full source code available), so it's possible to do wavelet compressed raw on a cell phone. If someone created an app to do this, would it trigger a lawsuit? Other than court fees, the biggest issue with lawsuits is having to pay the lawyers (small companies can simply declare bankruptcy and not pay any "damages" so loss risk is negligible, especially with truth on their side). What happens if one can prosecute/defend pro se (little or no lawyer fees beyond consulting) and testify truthfully in front of a jury with confidence against paid liars- who's going to be more convincing? Looks like the costs of patent suits is dropping: https://www.bna.com/cost-patent-infringement-n73014463011/ . There's plenty of people out there who could build this kind of app, e.g the Filmic Pro folks (and similar) and perhaps with Kickstarter et al funding and pro bono assistance get bogus patents invalidated for the benefit of society.

EDIT: I downloaded and built the Cineform codec on OSX (C/C++). It will build for iOS (vanilla C/C++). It should be possible to get at the raw captured pixels from the CMSampleBuffer, and pass the data to the Cineform encoder running in another thread (one of the two high-performance cores), then write to disk in another thread (using the other high performance core if it makes a difference (else can use one of the 4 lower energy cores). iPhone X and later have pretty fast ARM processors- not clear if fast enough to get 24fps. Certain parts can be re-written to run on the GPU- that would be easily fast enough. Also not clear if iOS will deliver raw frames fast enough (it should, but you never know until testing). When reviewing the docs, I noticed that ProRes 422 and 444 were also now supported. Curious if ProRes raw is coming to the phones- that would make this whole exercise obsolete (if the goal is just to get raw; wavelet is still superior for quality+efficiency (at least 2x better than ProRes (DWT vs. DCT)). Curious why Apple didn't do their own wavelet version given the superior math/capability/efficiency?
 
Last edited:
Gary T said:
I've seen a few of those R5's on Ebay. What kinds of issues would you look out for if you worried buying used?
Click to expand...

Not getting ripped off!

But seriously, you can ask the buyer how many hours on the clock, which he can tell you from the camera if he's the user, and make sure all 4 of the little hooks that attach it to the camera look good in the pictures. You can infer it could have been an issue since the R7 redesign added a support bracket across the top.
 
Gary T said:
I misspoke. If Red had sued them it would be a matter of public record. More likely they threatened to sue BMD much as they did Atomos before Atomos caved to their demands. Red did sue Sony for the compressed RAW in the F5 and F55 and that case was settled out of court. But you are right, it could be some other company.

I just don't get how you think a company that encrypts video files so the owners of the camera can't read them is more open than a company (Apple) that licenses their format to more than 100 products. Red doesn't license their format to any other camera vendor, and sues to keep other camera makers from delivering compressed RAW.

We're going to have to disagree here.
Click to expand...

I'm not actually saying that Red is more open, but rather that Red has a better reputation for how it treats its customers... which compared to Apple is a pretty low bar.
 
It's a shame that Cineform never got more traction.
With it's RGB/YUV and raw flavors, and pretty low computing needs, it would be an ideal codec for pretty much anything.
 
John Brawley said:
Rumours were always that REDCODE was licensed / a version of Cineform.

My reading of their patent claim when they went after Sony was they claim compressed RAW and the way they handle the YUV RGB transforms (different compression schemes for the Y and UV)

JB
Click to expand...

Do you have a link to that patent?

Raw has only a single plane of Bayer pattern RGB pixels. Raw is compressed in RGB Bayer form (e.g. RGGB), then in post is de-Bayered to per pixel RGB (via complex interpolation) after which a simple matrix transform converts pixels to YUV so UV (chroma: color) can store 1/2 (422) or 1/4 (420) color resolution where Y (luma) is stored full resolution (there are other (rarely used) variations as well).

It’s not clear how any of that prior art would be related to a camera patent for a company whose codec is wavelet compressed raw (no YUV pixels for any raw formats).
 
Postmaster said:
It's a shame that Cineform never got more traction.
With it's RGB/YUV and raw flavors, and pretty low computing needs, it would be an ideal codec for pretty much anything.
Click to expand...

Cineform software was a bit buggy. Now that GoPro has open sourced it, it’s pretty reliable. Maybe it will get more traction now? (Still a few years perhaps before AI deconstruction/reconstruction (not a form of compression) hits the market (making traditional compression obsolete)).

A bigger question is why not more wavelet use- camera hardware was cheaper to produce for DCT perhaps? DCT is still better for interframe/IPB (frame interpolation) compression.
 
jcs said:
Cineform software was a bit buggy. Now that GoPro has open sourced it, it’s pretty reliable. Maybe it will get more traction now? (Still a few years perhaps before AI deconstruction/reconstruction (not a form of compression) hits the market (making traditional compression obsolete)).

A bigger question is why not more wavelet use- camera hardware was cheaper to produce for DCT perhaps? DCT is still better for interframe/IPB (frame interpolation) compression.
Click to expand...

I read the Cineform open source as being free to use in software, but must be licensed if you put it in a camera, which is probably what caused BMD to drop CDNG (just speculation). I think I remember something from Adobe's site that said software=ok, hardware=license but I could be wrong.

As far as wavelet vs DCT, there aren't many ASICs made that use the wavelet compression, just plenty of MCU and ASIC designs have DCT engines built in. Not sure why, but I'm guessing licensing and compatibility with the rest of the world. No sens in recording to Cineform if the cell phones can't play it (poor example, but there you go).
 
Greg_E said:
I read the Cineform open source as being free to use in software, but must be licensed if you put it in a camera, which is probably what caused BMD to drop CDNG (just speculation). I think I remember something from Adobe's site that said software=ok, hardware=license but I could be wrong.

As far as wavelet vs DCT, there aren't many ASICs made that use the wavelet compression, just plenty of MCU and ASIC designs have DCT engines built in. Not sure why, but I'm guessing licensing and compatibility with the rest of the world. No sens in recording to Cineform if the cell phones can't play it (poor example, but there you go).
Click to expand...

Earlier in this thread I posted a section from the software license: the software itself is free (worldwide and perpetual). If there's a patent for use in cameras that blocks wavelets, it would be interesting to see the specific patent.

As for hardware and cost (profit): it could be memory related too. Wavelets are frame based which is comparatively a lot more memory than required for DCT working on much smaller macroblocks, so perhaps that was part of the issue? Now memory is relatively cheap and very powerful general purpose hardware is fairly low energy (multicore CPUs, GPUs, and vector processors).

I'm confident an iPhone X or later (or similar Android) can capture and play back wavelet compression. The question is, are there enough users in that market that would want raw video? For 99.99% of typical cellphone users, H.264/H.265 videos are plenty good and the comparatively tiny file sizes are more valuable than improved image quality and tweaking ability in post: most don't do any post work at all, except perhaps adding a filter on IG etc.
 
Don't need RAW to make it popular, just higher compression with better playback and fewer artifacts would be enough to push wavelet into phones.
 
Wavelet (currently) only works as ALL-I / intraframe (no interframe interpolation), so it cannot compete with H.264 and H.265 (about 2x more efficient that H.264) doing IPB (what most phones use) for quality+efficiency. 10-bit YUV (where UV is more compressed) wavelet is about 2x more efficient than ProRes 422 10-bit; haven't seen that on a phone yet- doesn't appear even Filmic Pro supports ALL-I H.26x? (only high bitrate H.26x from looking at the menu).
 
You must log in or register to reply here.
Back
Top