Page 8 of 8 FirstFirst ... 45678
Results 71 to 76 of 76
  1. Collapse Details
    #71
    Default
    How does the lossy work?

    On October 5, 2012, Adobe added lossy compression with version 1.4 of its DNG spec. DNG already had lossless, using a formula called Huffman. This new lossy way uses good old JPEG. In the spec it's limited to 8 bit, because at the time JPEG only did 8 bit. (Version 1.4 also added a second lossless formula, DEFLATE.)

    On January 20, 2014, the Independent JPEG Group released version 1.9 of JPEG, which added bit depths up to 12 bits per channel.

    On November 20, 2014, Blackmagic introduced lossy RAW, starting with a 3:1 ratio. Elsewhere I read that they base it on the aforementioned lossy DNG spec. But instead of limiting it to the spec's official 8 bits, they went with 12.

    My question is this. Before compression, how does lossy RAW split up the Bayer image? If you were to compress the whole image with JPEG, it would be inefficient. JPEG could do it, but it relies on swaths of color for savings. Meanwhile a raw Bayer image is guaranteed to have every pixel different from the one one right next to it.

    bayer.gif

    To me it makes the most sense to first extract all the green pixels, collapse the spaces in between, and JPEG-compress that image. Then do the same for red and blue. So a RAW compressed file would really be, in a sense, three images, each compressed separately. I have read that this is how Redcode does it. It just uses JPEG2000 instead of JPEG. In fact, the DNG spec itself seems to suggest this technique (see p. 20, Horizontal Difference X2).

    I just wasn't sure, because another article, talking about lossless compression, seemed to say that it just coalesced every other row, in effect splitting it into two different images, instead of three. It said that every other row has the same pattern, so the pixels would line up.

    Bayer_Pattern_Actual.gif

    But I think that it would still be much less efficient than cutting a Bayer image into three separate channels.


    ---

    Unrelated, I would like to see 10-bit log lossy compression, not just 12-bit.
    Last edited by combatentropy; 08-25-2017 at 09:58 PM.


    Reply With Quote
     

  2. Collapse Details
    #72
    Default
    @combatentropy:
    Blackmagic cameras compress all raw data as a single field. They do this in both lossless and lossy modes. Works ok for them in lossless (and yes, can be optimized, the nature of their log curve sort of hinders higher ratios), and in lossy they use the single field as a way to do bitrate control. slimRAW, on the other hand, splits channels in both lossless and lossy modes. The specification gives you leeway.
    I can't see any theoretical or practical need for nominal 10 bit log lossy. You can have any actual bitdepth below 12 by using 12-bit lossy and adjusting quantization accordingly. In fact, BM cameras (and Red cameras, and pretty much all cameras recording lossy raw) vary the true bitdepth all the time to achieve the desired constant data rate.


    Reply With Quote
     

  3. Collapse Details
    #73
    Senior Member
    Join Date
    Nov 2012
    Posts
    227
    Default
    @cpc:

    on my hexacore i4930k processor SLIMRAW only uses around 3% of CPU power - therefore, compression is rather slow. Is there a way to make Slimraw use more CPU-Power?


    Reply With Quote
     

  4. Collapse Details
    #74
    Default
    Quote Originally Posted by Fader8 View Post
    @cpc:

    on my hexacore i4930k processor SLIMRAW only uses around 3% of CPU power - therefore, compression is rather slow. Is there a way to make Slimraw use more CPU-Power?
    This is symptomatic for (very) slow storage. SlimRAW will use all the cpu power if your storage can keep up. What's your storage setup?


    Reply With Quote
     

  5. Collapse Details
    #75
    Senior Member
    Join Date
    Nov 2012
    Posts
    227
    Default
    Quote Originally Posted by cpc View Post
    This is symptomatic for (very) slow storage. SlimRAW will use all the cpu power if your storage can keep up. What's your storage setup?
    I now tried compressing from my RAID 0 (approx. 300-350MB) to a SATA SSD...now I get 13% CPU-Usage.


    Reply With Quote
     

  6. Collapse Details
    #76
    Default
    OK, this is a 4x improvement. You are probably not sustaining 300 MB/s throughput though. SlimRAW should be able to handle 700-800 MB/s on your CPU.
    Аn indicator of the current bottleneck during processing (input storage, output storage or CPU) is on the ToDo list.

    (And don't forget to update, version 1.8.2 was released yesterday: http://www.slimraw.com/download.html )


    Reply With Quote
     

Page 8 of 8 FirstFirst ... 45678

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •