MJPEG Encoder Research, no questions here

Eftegarie said:
We achieved AVCHD 61 mbit as averaged(!) output: visibly sharper than MJPEG.
Click to expand...

Very interesting. Will have to try with shorter GOPs. Have you ever tried shooting some of the charts (PappasArts or Crunchy's) or with 25FPS? I suppose that you tested with 24pn (native) option?

Added: What does it mean that video "hicks? You said "the video hiks (skips some frames once a second orso) as if the choke limitation of 70mbit doesnt allow to dump all frames...". Have you checked video frame-by-frame and does it still "hicks"? Have you checked it with Stream Parser? Please, can you upload somewhere one of the "hicks" videos?
 
Last edited:
Interesting that extremely low GOPs are possibly working now. Before a setting of 1 or 2 was pretty much guaranteed an instant crash (was this native mode?).
 
Last edited:
Eftegarie said:
We achieved AVCHD 61 mbit as averaged(!)
Click to expand...
Are you sure there is no mistake here ?
I use similar settings (only on a PAL camera) which result in picks at 55Mbps ....
Can you send a screen shot from Gh13streamparser for that kind of file ?
If you switch to frames mode you can see the average bit rate.

Sassi
 
modulr said:
Interesting that extremely low GOPs are possibly working now. Before a setting of 1 or 2 was pretty much guaranteed an instant crash (was this native mode?).
Click to expand...
Even when the camera did not crash, I've spotted frame errors in AVCHD files produced with GOPs of 4 frames or less. These appear to be due to internal codec glitches rather than SD card write speed limitations.
 
I have test shots going from GOP 14 to 7 to 5 to 2 .
In the parser program you can see I frames increasing , and file sizes growing....Big change occurs at GOP5 as pic starts to darken and lose dynamics then at 2 darkens again stream is then odd looking and flattened

All exact same scene no movement in frame file sizes go ~ 21 meg then,75 meg then 129 meg then 158 meg all for the same 40 sec capture I think at GOP 5 we are seeing codec breakdown as this level of low compression is not designed into this hardware platform..
Yes it will do it but not efficiently.. and the picture suffers for it . So far my research seems to want not to push GOP lower than 7 at this point ...GOP 5 seems to make the Pic crush blacks and lose midtones ...more experiments needed .

also bought and tested panasonic gold and sandisk extreme 8gig units

based on cartridge burst mode tester supplied on this forem my results were:

panasonic gold :

56.3 Mbs at 32 k burst
76.6 64k
87.2 128k
83.2 256K

sand* extreme III

80.3 Mbs 32k
102 64k
112.4 128k
112.0 256k

so the sandisk is a lot faster than the panasonic

this is reflected in My Mjpeg resting at 100 to 176 Mbs the panasonic just fails and the sandisk keeps goin on I must say that the 100 Mbs files are also really pushing the hardware as well as blacks seem to be crushed and there is a general sense of stressing the unit and noise generation .. 50 to 75 mbs seems to be about right ..

I am now using Zeis contax G 45 for testing and it will be the definitive unit for clarity as for 35 mm it is considered one of the sharpest lenses wide open ever produced ...so it is overkill for our 1080 p tests...

cheers
Larry
 
Dear Tester13 & Power to the People Crew,

We have tested [Sensor mode 4] set to nr 7 (23,97 fps) and tried out the other [Sensor mode 4 adjustments], but they seem to have NO effect.

All other bitrates and preset bitrates unticked, @ 24 fps Full HD AVCHD

See the progress in our office at http://www.aster.nu/gh1.php

Search for "Sensor Mode 4 Adjusments"

w=2200 | h=1239 | rw=960 | rh=844
Not a single Effect!?

w=2200 | h=1239 | rw=960 | rh=844 | sensor h cut = 500 | sensor w cut =0
Not a single Effect!?

Any ideas for next tests? Thanks & Goodluck and Good health to you all.
 
I tried modulr's settings and on the new ptool I'm having issues, the lower portion of the video is static noise, then blue and red lines fill the screen vertically. I used these setting before on an earlier version of ptools and had none of these problems. I have sensor mode 4 and number 7 selceted, I have tried with and without 422 1080 and no difference.

If anyone else has experienced this and know what is causing it (most likely the sensor mode but I think it is helping make my avchd 24p more stable) I would love to hear from you, I'm going to try and change the table values to 24 and hopefully that should work.

If needed I can post a sample of it tomorrow.

have a good one.

edit: yup should have waited a few minutes till I did my next test, seems the sensor mode 4 setting 7 and mjpeg do not like each other as soon as I turned it off all the mess went away
 
Last edited:
HD resolution in WVGA video mode?

HD resolution in WVGA video mode?

In a discussion on MJPEG frame size patching, dvxuser johnnym recently pointed out the following post, where Vitaliy mentioned that the VGA mode frame size might be patchable as well:

http://www.dvxuser.com/V6/showthrea...-concentrate&p=2107674&viewfull=1#post2107674

Increasing the resolution of the MJPEG VGA mode would be very useful, since it scans the image sensor at a 4:3 aspect ratio. If we could set the VGA frame size to 2160x810, we could use it with a 2x anamorphic lens to produce 2.67:1 widescreen videos that would not require further post-processing. (Assuming that the Q1-Q4 patches would increase its bitrate as well.)

Any word on how feasible this kind of patch would be?
 
Last edited:
Just to add on to what lpowell brought up...A majority of easier to find anamorphic lenses are 2x so this would make being able to film in cinemascope much more feasible. Currently you either have to crop a lot of your image off or search for a 1.33x (which I have yet to find one available for sale). Couldn't we also use a 1440x1080 framing with a 4:3 to end up with a 2880X1080? I just read that link and a possible 1600x1200 or 3200x1200 with a 2x...I mean that is just INSANE!
 
Hey guys. I'm planning to do some rigorous testing of boosted MJPEG quality on my GH1. Figured I should chime in here.

--------------------

Fwiw...my background is in software development since the early 1980s, on & off, ranging from assembly language on various small processors to client-server enterprise web-app development. I've been doing multimedia/digital video work, again on & off, since the early 1990s (QuickTime 1.0 and MPEG1 days). I do NOT have experience with reverse-engineering, let alone tackling a highly integrated system like a consumer camera (huge props to Vitaliy for his skills and efforts!). I DO have a good general sense of how low-level code hangs together. And I take a very rigorous approach to testing. So hopefully I can be of some help here. :)

--------------------

First I'd like to confirm what is known so far and ask a few questions...


In post #5, Vitaliy made the following comments & educated guesses:

1) The Q values represent some kind of estimated quality level.

2) The T values are indices into a 1KB table of byte values.

3) The four Q-T pairs represent four levels of progressively lower quality that the camera falls back to when it cannot keep up with the bitrate.

4) The quantization matrices are all implemented in hardware (not available to firmware inspection).


From what I can tell, most of the testing so far has taken these points as hard fact, poked at the Q&T values, and looked at the quant tables in the first frame of video as an potential indicator of quality. Is that more-or-less accurate?

But...we still don't have a firm idea of what the Q&T values actually are, and how they work together to change the video bitrate or quality. Note: I look at bitrate and quality separately; they are not always correlated. ;)

According to Vitaliy's post #5, the T values are just indices. Someone graphed them earlier in this thread. That seems kinda like...graphing the box numbers in a post office, to attempt a correlation with the type of mail in each box.

Have you guys looked ~inside~ the boxes, i.e., at the values in the 1KB table that these indices reference? Vitaliy, you can see that table in the firmware, right? Could you post those 1024 values here? Then we could look for more meaningful correlations and patterns.

If I've misunderstood any of this stuff, or if I missed technical details from other threads, please correct me. :)

--------------------

My test methodology is (open to your comments):

1) Construct a static test scene with fixed lighting, and with areas of high detail, solid black, and smooth gradients. This static test scene, shot from tripod, will allow a meaningful comparison of different settings.

2) Shoot some still JPEG images of the test scene to establish a baseline of maximum image quality from the GH1. I expect these shots will be higher quality than even the best MJPEG frames...otherwise why would the GH1 even have a shutter? :)...but still it gives us something to compare.

3) Shoot a 30-second MJPEG clip of the test scene with specific E1-E4 hack settings.

4) Play each clip at 1/4 speed to check for dropped frames or other hiccups.

5) Discard the first 20 seconds of each clip. This will ensure that a quality boost is not just a temporary initial effect while the buffers are empty. Dump the next 10 seconds of footage as a series of 300 JPEG images.

6) Using jpegsnoop, extract the "Approx quality factor" of 1 frame every 15 frames (20 frames total) to see if/how quality varies over 10 seconds of recording.

7) Using jpegsnoop, extract the "Approx quality factor" of 30 adjacent frames to see if/how the quality varies frame-to-frame over 1 second of recording.

8) Take close-up crops of "difficult" (detailed or dark) areas in the images, to VISUALLY compare quality between frames. IMO this is the most important testing step, since we have no guarantee that the Q&T settings affect ONLY the quant matrix.

9) Graph the quality factor and byte-size of each frame, across each of these sets of frames.

10) Repeat 3-9 with different settings for Q1-Q4 and T1-T4.

I welcome any comments on this methodology!

All of my testing will be in 720p30 4:2:0, because:
a) It's what I shoot. :)
b) The smaller amount of data (compared to 4:2:2) means I can test a larger range of quality settings without card speed errors.
c) MJPEG frames can be compared directly against still JPEGs, which also use 4:2:0.
d) Focusing on a narrow set of parameters will make it easier to track down the Q-T-quality relationship.

--------------------

My first tests will be:

1) Change the E1 values to use factory E2 values, then to use factory E3, then E4. If Vitality is correct that these four levels represent fallback positions to avoid buffer overflow, each of these tests will show a lower bitrate. This seems like the obvious first test, but I didn't see that anyone has done it yet. If you have already done this, please let me know!

2) Try the high-bitrate settings posted by other testers so far, BUT look beyond the first frames, look at the quality over time, and do VISUAL comparisons, to see if the higher-bitrate results have higher quality as well.

Then I'll start poking with different settings, ideally with a better understanding of Q&T values. A copy of the 1024-byte table referenced by the T values would help to guide this research.

--------------------

One last idea/request (for now):

Vitaliy, would it be possible to hack the Rec Quality menu settings so that all four options (HD, WVGA, VGA, QVGA) record the SAME resolution (eg 720p30, set in the PTool), but with different E1-E4 values?

This would make testing MUCH faster, since we could test 4 different E1-E4 settings with each firmware reflash. Firmware reflashing seems to be the worst bottleneck for testing.

This would also allow us to switch between settings on the fly, in real usage. i.e., Use 65+Mbps when we need highest quality, or switch back to the stock 32Mbps when we need long clip times or in-camera playback, without reflashing the firmware.

That would be AWESOME.

------------------------

Whew OK that's it for now. I hope some of you folks are still around and interested in this stuff? :)

J
 
First - THANK YOU to Mr. Kisalev for producing the GH1 hack tool, and to lpowell, modulr, PappasArts, and others for finding some good working sets of MJPEG hack values. You guys rock!!!

OK, so I finally ran some tests last weekend. Here is what I tried and found.


--------------------------------------
SETUP
--------------------------------------

Subject -- Static full-frame close-up on a fir tree, branches & needles, full sunlight. On a scale of 1-10 where "1" is a flat white card and "10" is random pixel noise, this is probably a "7". More detail than average, but not unusual for a nature scene. Grass or feathers or fur would have even more detail.

Lens & exposure -- Sharp focus across the entire frame. Canon 50mm/f1.4 lens, stopped down to f16, shutter 1/80, iso 100, heavy tripod.

Automatic features & filters -- All disabled, so they won't change the image from frame-to-frame, or steal CPU cycles from the encoder. Manual focus, manual exposure, iExposure OFF, white balance DAYLIGHT, film mode STANDARD, shoot w/o lens ON.

Memory card -- Transcend 16GB Class 10.

Video mode -- 720p30, 4:2:0.


--------------------------------------
HACKS
--------------------------------------

PTool v3.51d (291110)

"Version increment" and "MJPEG Compression" values only.

Values:

Code: 
factory (unhacked) .... 128, 133, 110, 146, 100, 150, 92, 155

lpowell ............... 280, 4, 250, 10, 225, 24, 200, 48

modulr ................ 400, 4, 300, 4, 250, 4, 188, 4

PappasArts ............ 384, 24, 330, 24, 300, 24, 276, 24

factory e3 ............ 100, 150, 100, 150, 100, 150, 92, 155

factory e4 ............ 92, 155, 92, 155, 92, 155, 92, 155



--------------------------------------
RESULTS
--------------------------------------

Each clip was shot for at least 30 seconds (900 frames), then I dumped the clips to individual JPEG files (no transcoding) for inspection.

Code: 
Frames     Size      Rate       Quality
xxx-xxx    xxx KB    xx Mbps    Q=xx

"Size" is the size in KB of each JPEG frame.

"Rate" is bitrate at 30fps, calculated from frame size.

"Quality" is the averaged "Approximate quality factor" reported by JPEGsnoop.

Code: 
Factory
001-900    126 KB     31 Mbps    Q=19

lpowell
001-030    516 KB    127 Mbps    Q=93
031-900    288 KB     71 Mbps    Q=74

modulr
001-015    781 KB    192 Mbps    Q=97
016-060    474 KB    116 Mbps    Q=91
061-885    346 KB     85 Mbps    Q=82
      ***write-speed error***

PappasArts
001-015    388 KB     95 Mbps    Q=88
016-045    334 KB     82 Mbps    Q=84
046-900    298 KB     73 Mbps    Q=80

Factory E3
001-900    125 KB     31 Mbps    Q=19

Factory E4
001-900    115 KB     28 Mbps    Q=19


Observations:

1) The non-factory hacks all stepped-down quality within the first 2 seconds.

2) Quality step-down, or write failure, only occurs on 1/2 second boundaries (multiple of 15 frames).

3) None of the settings stepped down more than twice. The modulr settings stepped down twice before failing. I assume modulr's E4 settings were still too aggressive to keep up. Must test to confirm.

4) Using the factory E3 or E4 settings for E1 and E2, made a VERY SMALL impact on bitrate and quality. Less than 10% difference between E1 and E4. Unexpected. This range seems too small to make a real-world difference.


--------------------------------------
SUBJECTIVE QUALITY
--------------------------------------

I pixel-peeped the frame grabs from these tests, and came to the following conclusions:

1) Stock 31 Mbps rate (126 KB/frame) looks as awful as you might guess from the quality=19 reported by JPEGsnoop. You can see extensive macroblocking in the dark areas of the frame, without even magnifying. If you magnify the image, it quickly degenerates into noise, with many off-tint macroblocks.

2) lpowell's 71 Mbps rate (288KB/frame) gives a DRAMATIC increase in picture quality over the stock settings. Awesome. The massive macroblocking disappears.

3) Higher bitrates, even the whopping 192 Mbps for the first 1/2 second with modulr settings, give only a SMALL increase in quality. You have to pixel peep very intensely to see any improvements.

I think the reason is: Frames sent to the encoder in video mode are already heavily blurred by the pixel-binning and noise reduction that shutterless video requires. So, quality levels beyond 70-80 do not help much. The details are simply not there to be compressed. Sharpening the image (via film modes) would probably increase the difference between these settings. BUT we can do better sharpening in post, and get much longer recording times.


--------------------------------------
CONCLUSION
--------------------------------------

Constant 70-80 quality is best for MJPEG on the GH1. We can achieve this quality by approximately doubling the factory bitrate.


--------------------------------------
NEXT STEPS & REQUESTS
--------------------------------------

1) I will tweak the Q values to see if I can get a constant 70-80 quality on all frames, and then test it with longer recording times and kit lens autofocus.

2) I would like to play with the T indices, but I don't want to shoot "in the dark" with absolutely no idea what these values do. It would be most helpful, if you guys could post other T settings you have already tested, and how you settled on T settings that work. modulr already explained his tests of T values 1-4. Others like lpowells 4,10,24,48 are a total mystery to me.

3) If we could hack four different MJPEG settings at once (using HD, WVGA, VGA, QVGA options) it would radically speed up testing. The 2-minute reflash is the biggest bottleneck to testing.

4) The 2GB MJPEG limit is the other big issue. With 2x factory bitrate, we can record only ~4 minutes of video. Vitaliy - Is it possible to hack the firmware so MJPEG recording automatically starts a new clip, when the 2GB limit is hit? Even if it missed a few frames, it would be great!


Thanks,
J
 
Last edited:
Thanks for conducting these tests, jaydil, the results are very interesting. The only thing I'd point out is that I fine-tuned my MJPEG settings to optimize 4:2:2 color depth, which increases the amount of data on each row of pixels by about 30%. While these settings will also work fine in 7:2:0 color depth, they're a bit conservative for a 1280x720 frame size. At 7:2:0, I've successfully used this patch with a frame size as large as 1920x810.

I am surprised to see a bitrate over 100Mbps for the first 30 frames of video, though most Class 6 SD cards are able to handle this kind of burst speed. It's most likely due to the T1 setting of "4" in my patch, which is the table that uses the lowest practical quantization factors in compressing the image. However, the Q1/T1 settings are actually tuned to maintain high bitrates in dimly-lit scenes, which is a type of performance this test wouldn't reveal.

I'm pleased to see a long-term Quality Level of 74 for your well-lit, sharply-focused, highly-detailed scene. This is very close to the Q75 level I was aiming for, which is the default "High Quality" setting used by standard JPEG encoders. By limiting high-detail images to Q75, I was able to boost the Quality Level of low-light images to over Q95 while minimizing the risk of "write-speed errors".

I'd be most interested to see a comparison of this set of MJPEG patches using a low-light scene with sharply-focused color gradients rather than busy details. If you have time to do this, it may show how well each patch can maintain high bitrates in under-exposed conditions.
 
Thanks lpowell.

From my tests, it appears that your Q2/T2 settings are totally sufficient for a high detail, sharp-focus scene. Probably overkill (wasted bandwidth) for a low-detail, low-light, soft-focus, or shallow-DOF scene. But I can test that kind of scene to see what happens. Can you suggest a good/common subject and exposure for this?

Also I still want to know: What other T indices did you try? How did you settle on 4/10/24/48 for these values?

Did you test this against other settings, or were these settings just a random shot in the dark that happened to work OK, or....?

Thanks,
J
 
The four sets of Qn/Tn entries specify the quality factor and quantization matrix used by the encoder at each of its four internal quality levels. MJPEG coefficient scaling appears to be proportionally related to the quality factor. With each block of coefficients, the codec first attempts to encode it with Q1/T1 settings. If this exceeds its allocated bitrate budget, it retries with Q2/T2, then Q3/T3, and finally resorts to Q4/T4.

Factory settings specify a decreasing series of quality factors for Q1-Q4 and a mildly increasing series of quantization matrices for T1-T4 (which increases the coarseness of the quantization). Previous attempts to hack the MJPEG encoder had revealed that Table 4 was probably full of 1's, as it was capable of producing JPEG Quality levels of 100. However, this table produces inefficient compression of high-frequency spatial details, generating excessive bitrates for images with heavily-aliased edge effects. For this reason, I use Table 4 only in the Q1/T1 setting, in order to preserve as much of the dark details of dimly-lit images as possible.

For Q4/T4, I determined that a quality factor of about 200 would restrain the maximum average bitrate for highly detailed images to around 65Mbps. I then used a binary seach to find the lowest table index that produced reliable recordings without write-speed failures, and iterated the quality factor to arrive at 200. Q2/T2 and Q3/T3 were then interpolated and tested on a variety of moderately detailed and/or illuminated scenes to confirm that the bitrate did not sag in the middle of the encoder's complexity range.
 
Last edited:
lpowell said:
I then used a binary seach to find the lowest table index that produced reliable recordings without write-speed failures
Click to expand...
Thank you lpowell!!! This is good info.

What range of table indices did you test? The ptool allows values from 0-1024 (I assume this should really be 0-1023, or 1-1024). Did you search that entire range, or a subset? And did the T values that you tested all work OK, and converge on your desired bitrate as you hoped...or were the results more random?

Many thanks for any more info you can provide. I suppose I could just leave it alone and play with the Q values at this point, but I'm curious to figure out exactly what the T values do.

It would be awesome if Vitaliy could post the table of byte values referenced by these T indices. That would give us a much better idea of what they really represent.

J
 
jaydil said:
Thank you lpowell!!! This is good info.

What range of table indices did you test? The ptool allows values from 0-1024 (I assume this should really be 0-1023, or 1-1024). Did you search that entire range, or a subset? And did the T values that you tested all work OK, and converge on your desired bitrate as you hoped...or were the results more random?

Many thanks for any more info you can provide. I suppose I could just leave it alone and play with the Q values at this point, but I'm curious to figure out exactly what the T values do.

It would be awesome if Vitaliy could post the table of byte values referenced by these T indices. That would give us a much better idea of what they really represent.

J
Click to expand...

It would be great if we could get a table to understand ptool values. it would sure help us understand more of what the values are while tweaking away. I'm up to over 500 versions of AVCHD patches as is and still slightly confused.
 
rigs said:
It would be great if we could get a table to understand ptool values. it would sure help us understand more of what the values are while tweaking away. I'm up to over 500 versions of AVCHD patches as is and still slightly confused.
Click to expand...
I think the AVCHD hacks are well described in the ptool popup help. The MJPEG values are the most mysterious.

Vitaliy said that the "table" values are indices into a table of byte values, but he has not provided that table, so everyone who has tested these values is really just shooting in the dark.

Vitaliy, are you still alive?? Pretty please, post the table values for us?

lpowell, are you still alive?? Pretty please, post the range of values that you tested, so we know what "shots in the dark" have been taken already?

:)
J
 
For background information on JPEG quantization matrix tables, see the following links:

http://en.wikipedia.org/wiki/Quantization_matrix#Quantization_matrices
http://en.wikipedia.org/wiki/JPEG

The GH1 firmware appears to contain 1024 hard-coded quantization matrices in ROM, selectable by the T1-T4 patch settings. Tables 0-3 produce blank results and are useless. Table 4 appears to produce the least amount of quantization quality loss, and I suspect it's full of one's. Higher table indexes appear to produce increasing amounts of both data compression and quantization quality loss. The highest numbered table used by the GH1's MJPEG factory settings is Table 155, and I haven't tested any tables higher than that.
 
You must log in or register to reply here.
Back
Top