D-Movie DQT

[bearing]

I found a software called JPEGsnoop. By loading an MJPEG into it I could get the Discrete Quantization Tables used in D-Movie mode. Quality factor is about 75 percent.


Here they are:

  *** Marker: DQT (xFFDB) ***
  Define a Quantization Table.
  OFFSET: 0x000239C0
  Table length = 132
  ----
  Precision=8 bits
  Destination ID=0 (Luminance)
    DQT, Row #0:   4   6   5   8  12  20  26  31 
    DQT, Row #1:   6   6   7  10  13  29  30  28 
    DQT, Row #2:   7   7   8  12  20  29  35  28 
    DQT, Row #3:   7   9  11  15  26  44  40  31 
    DQT, Row #4:   9  11  19  28  34  55  52  39 
    DQT, Row #5:  12  18  28  32  41  52  57  46 
    DQT, Row #6:  25  32  39  44  52  61  60  51 
    DQT, Row #7:  36  46  48  49  56  50  52  50 
    Approx quality factor = 74.94 (scaling=50.12 variance=10.81)
  ----
  Precision=8 bits
  Destination ID=1 (Chrominance)
    DQT, Row #0:   4   9  12  24  50  50  50  50 
    DQT, Row #1:   9  11  13  33  50  50  50  50 
    DQT, Row #2:  12  13  28  50  50  50  50  50 
    DQT, Row #3:  24  33  50  50  50  50  50  50 
    DQT, Row #4:  50  50  50  50  50  50  50  50 
    DQT, Row #5:  50  50  50  50  50  50  50  50 
    DQT, Row #6:  50  50  50  50  50  50  50  50 
    DQT, Row #7:  50  50  50  50  50  50  50  50 
    Approx quality factor = 74.97 (scaling=50.06 variance=11.27)
 
*** Marker: SOF0 (Baseline DCT) (xFFC0) ***
  OFFSET: 0x00023A46
  Frame header length = 17
  Precision = 8
  Number of Lines = 720
  Samples per Line = 1280
  Image Size = 1280 x 720
  Raw Image Orientation = Landscape
  Number of Img components = 3
    Component[1]: ID=0x01, Samp Fac=0x21 (Subsamp 1 x 1), Quant Tbl Sel=0x00 (Lum: Y)
    Component[2]: ID=0x02, Samp Fac=0x11 (Subsamp 2 x 1), Quant Tbl Sel=0x01 (Chrom: Cb)
    Component[3]: ID=0x03, Samp Fac=0x11 (Subsamp 2 x 1), Quant Tbl Sel=0x01 (Chrom: Cr)

I was curious as to whether tables are custom made at Nikon or some sort of standard tables. I saved a JPEG in Paint Shop Pro at 75 quality to compare to. Tables are identical apart from the DC components which are more compressed in the PSP JPEG. I guess Nikon took standard 75% quality tables and changed the DC components to 4.

*** Marker: DQT (xFFDB) ***
  Define a Quantization Table.
  OFFSET: 0x00000014
  Table length = 67
  ----
  Precision=8 bits
  Destination ID=0 (Luminance)
    DQT, Row #0:   8   6   5   8  12  20  26  31 
    DQT, Row #1:   6   6   7  10  13  29  30  28 
    DQT, Row #2:   7   7   8  12  20  29  35  28 
    DQT, Row #3:   7   9  11  15  26  44  40  31 
    DQT, Row #4:   9  11  19  28  34  55  52  39 
    DQT, Row #5:  12  18  28  32  41  52  57  46 
    DQT, Row #6:  25  32  39  44  52  61  60  51 
    DQT, Row #7:  36  46  48  49  56  50  52  50 
    Approx quality factor = 74.75 (scaling=50.51 variance=0.81)
 
*** Marker: DQT (xFFDB) ***
  Define a Quantization Table.
  OFFSET: 0x00000059
  Table length = 67
  ----
  Precision=8 bits
  Destination ID=1 (Chrominance)
    DQT, Row #0:   9   9  12  24  50  50  50  50 
    DQT, Row #1:   9  11  13  33  50  50  50  50 
    DQT, Row #2:  12  13  28  50  50  50  50  50 
    DQT, Row #3:  24  33  50  50  50  50  50  50 
    DQT, Row #4:  50  50  50  50  50  50  50  50 
    DQT, Row #5:  50  50  50  50  50  50  50  50 
    DQT, Row #6:  50  50  50  50  50  50  50  50 
    DQT, Row #7:  50  50  50  50  50  50  50  50 
    Approx quality factor = 74.74 (scaling=50.52 variance=0.19)
 
*** Marker: SOF0 (Baseline DCT) (xFFC0) ***
  OFFSET: 0x0000009E
  Frame header length = 17
  Precision = 8
  Number of Lines = 720
  Samples per Line = 1280
  Image Size = 1280 x 720
  Raw Image Orientation = Landscape
  Number of Img components = 3
    Component[1]: ID=0x01, Samp Fac=0x22 (Subsamp 1 x 1), Quant Tbl Sel=0x00 (Lum: Y)
    Component[2]: ID=0x02, Samp Fac=0x11 (Subsamp 2 x 2), Quant Tbl Sel=0x01 (Chrom: Cb)
    Component[3]: ID=0x03, Samp Fac=0x11 (Subsamp 2 x 2), Quant Tbl Sel=0x01 (Chrom: Cr)

We can also compare to still image tables (Fine quality). These are significantly less compressed with a highest factor of 3 compared to above factors of over 50.

*** Marker: DQT (xFFDB) ***
  Define a Quantization Table.
  OFFSET: 0x0000A5EC
  Table length = 132
  ----
  Precision=8 bits
  Destination ID=0 (Luminance)
    DQT, Row #0:   1   1   1   1   1   1   1   2 
    DQT, Row #1:   1   1   1   1   1   2   2   2 
    DQT, Row #2:   1   1   1   1   1   2   2   2 
    DQT, Row #3:   1   1   1   1   1   2   2   2 
    DQT, Row #4:   1   1   1   2   2   3   3   2 
    DQT, Row #5:   1   1   2   2   2   3   3   3 
    DQT, Row #6:   1   2   2   2   3   3   3   3 
    DQT, Row #7:   2   3   3   3   3   3   3   3 
    Approx quality factor = 98.02 (scaling=3.95 variance=3.57)
  ----
  Precision=8 bits
  Destination ID=1 (Chrominance)
    DQT, Row #0:   1   1   1   1   3   3   3   3 
    DQT, Row #1:   1   1   1   2   3   3   3   3 
    DQT, Row #2:   1   1   2   3   3   3   3   3 
    DQT, Row #3:   1   2   3   3   3   3   3   3 
    DQT, Row #4:   3   3   3   3   3   3   3   3 
    DQT, Row #5:   3   3   3   3   3   3   3   3 
    DQT, Row #6:   3   3   3   3   3   3   3   3 
    DQT, Row #7:   3   3   3   3   3   3   3   3 
    Approx quality factor = 98.39 (scaling=3.22 variance=0.43)
 
*** Marker: SOF0 (Baseline DCT) (xFFC0) ***
  OFFSET: 0x0000A672
  Frame header length = 17
  Precision = 8
  Number of Lines = 2848
  Samples per Line = 4288
  Image Size = 4288 x 2848
  Raw Image Orientation = Landscape
  Number of Img components = 3
    Component[1]: ID=0x01, Samp Fac=0x21 (Subsamp 1 x 1), Quant Tbl Sel=0x00 (Lum: Y)
    Component[2]: ID=0x02, Samp Fac=0x11 (Subsamp 2 x 1), Quant Tbl Sel=0x01 (Chrom: Cb)
    Component[3]: ID=0x03, Samp Fac=0x11 (Subsamp 2 x 1), Quant Tbl Sel=0x01 (Chrom: Cr)

The BASIC quality setting for stills is using the the same tables as D-movies.

 

[buildyo]

Very interesting bearing. What I don't understand is that the D-movie frames don't look like JPEGs compressed at 75% quality which is actually a quite good quality. I wished it would look like this but the D-movie frames look much, much worse. Are you sure the numbers are correct? 98% quality for the BASIC JPEGs is almost lossless. I doubt that this is correct or I don't understand what you say.

Check this out:
The D90 produces about 2MB/s so one frame weighs roughly 80 KB.
2000KB/s / 24 frames = 83 KB/frame

To save a 1280x720 photo as a 83KB JPEG file I have to set the quality setting very very low in Photoshops Save for Web Dialog.

 

[bearing]

I think the images are blurry before they are compressed. Don't know why. It has probably got to do with how the sensor Bayer pattern is read and converted to YUV.

Regarding the DQT, I've done some more testing now. The tables are variable. Simple scenes, or out of focus scenes gets 75% quality. When images gets more detailed and file sizes approaches 133 kB the quality decreases. I took some detailed pictures and JPEG quality went down to 35%. That may be due to my SD-card though. Its not an SDHC. 133 kB/frame * 24 fps is 3 MB/s which is the limit for SD-cards(I think). I will buy an SDHC-card class 6 and see if there is a difference.

98% is on the fine setting. Basic is 75%. However, they may be variable as well.

 

[buildyo]

II will buy an SDHC-card class 6 and see if there is a difference.

Save your money unless you want faster transfer to the computer. I use a Sandisk Cass 6 card with 20MB/s and the "quality" is every bit the same. Nikon could have written into the manual that the 720p mode requires a class 4 or 6 card but because they don't say anything about the card speed it looks like they've chosen the poor data rate to work with any ordinary SD card.

It's hard to understand what goes on in the heads of those people that decide on camera features. If I where Nikon I would be ashamed to release a DSLR with a movie mode that is inferior in terms of compression quality to that of some point and shoots.

 

[bearing]

Here are two JPEG:s. One is extracted from a D-movie stream. The other one is a still taken at the same time (shutter button was pushed while recording) (from a tripod). The still has been resampled to 1280x850 and then cropped to 1280x720. Since D-movie mode chose a quality of 54, the still is saved with that same quality.

I think this proves that the JPEG compression (even though it's high) is not responsible for the softness in images. Actually, difference between the uncompressed resized still and this highly compressed one is barely visible. The big difference is between the D-movie image and the still. It's even visible in these small previews!

For today, my money is saved. Measured write speed to 4.5 MB/s, so, that should not have been the the limit. The camera is probably programmed to limit data rates above 3 MB/s then.

 

[buildyo]

Yes, that's exactly the difference we all see. Very good comparison. Thanks. Here's mine:
http://www.dvxuser.com/V6/showthread.php?t=148666
Naiv as I was I first saw the data rate and thought ok it's not great but still "medium JPEG quality" and bought the D90. I didn't realize until I had it in hands that there are additional chances to mess up the movie quality. Nikon did a great job including them all in their D90 :crybaby:

 

[Nitemare]

If the data rare is only damped by the SD cards, then a firmware update or hack could make the life of class 6 card owners alot better...

 

[hdkhang]

I think the image quality is lower in video mode as the bayer interpolation is probably optimised for speed over quality. If you use liveview mode and zoom right in, the quality is pretty poor.

If Nikon implemented something similar to Canon's sRaw mode, that might help.

This would mean binning of pixels to reduce the amount of processing needed to do the bayer interpolation. If this happens, then we may get higher quality video.

If they decide to abandon D90 users, I do hope that at least their next implementation (hopefully in a full frame cam) is more oriented towards quality.

 

[Alpha Beta Omega]

I open alot of the clips I've made and watched the bitrate in each clip.

It's obvious the bitrate has a great span. Between 6000/kpbs and 32000/kbps.
All clips with lots of black areas makes the bitrate in the 6000-8000 region where outdoor clips in good light generell in the 10000-15000 range. Some clips reach in the 25000-32000 area.

Why? I guess it's that JPEG compression that discards black as nothing and preserve bits.