Possible breakthrough on Native 24p!

C

cbrandin

Well-known member
I've been digging into the GH1 firmware code and found that overall bitrate appears to have a big effect on the maximum speed that flash memory writes can occur. I’ve always wondered why the write speed error comes up in AVCHD mode when clearly the data rates are well below what the card can handle. I think I found the answer.

When the write speed error comes up it’s because the calculated maximum write rate has been exceeded – whether the card can handle more or not. I tried the Lpowell settings but changed two things: I doubled the overall bitrate to 86,000,000 and I checked Native 24p. Upping the overall bitrate nearly doubled the calculated flash write rate (i.e. the fastest write rate the camera attempts to write to flash). It appears to be working! I’m not getting any lockups and the clips look great. I tested this with highly detailed scenes, fast shutter speeds, everything I normally do to break the codec and it seems to be standing up.

We probably need to do some fine tuning and more testing (hint, hint, Lpowell), but so far it’s looking pretty good. I ran my tests with a Transcend class 10 card.

Chris
 
Last edited:
Lol. Right when I thought I had a pretty good setting , I found this thread. I'll test this out as well.
 
The theory behind this is that Native 1080 24p frames are twice as large as 720p and wrapped 1080p frames. The overall bitrate affects the instantaneous maximum write speed. With frames twice as large I surmised that the instantaneous write speed would have to be correspondingly faster. The video bitrate keeps the average bitrate over time to a manageable level.

Lets hope this holds up.

Chris
 
My latest theory was that it is not bitrate at all, but rather time related constant (being multiplier of 90000 ticks) :-)

May be it is good to look at all 18.000.000 locations and check routines logic carefully.
 
Yeah, that makes sense to me. At first I thought it was buffering too, but now I agree, it has more to do with transferr rate and write frequency. Youll notice that the pre-calculated minimum interval between writes is much smaller (the first four bytes of each 192 byte packet are the internal time stamp and the difference between them is the write interval) with the overall bitrate set high. The interval between packets is always a multiple of this pre-calculated number. Thus, the minimum interval absolutely limits the maximum frequency that packets can be written.

Chris
 
There are still a few things about the Native 24p streams that are a little strange - like an extra P frame every now and then and an empty frame also. It doesn't seem to affect how the clips look, though, so I'm not sure what to make of it. Also, I've seen some of this with 720p clips. It wouldn't be visible with wrapped clips because of the pulldown, etc... We might just be seeing artifacts that always existed and aren't related to the hack.

Chris
 
cbrandin said:
There are still a few things about the Native 24p streams that are a little strange - like an extra P frame every now and then and an empty frame also. It doesn't seem to affect how the clips look, though, so I'm not sure what to make of it. Also, I've seen some of this with 720p clips. It wouldn't be visible with wrapped clips because of the pulldown, etc... We might just be seeing artifacts that always existed and aren't related to the hack.
Chris
Click to expand...

Most probably this strange things happen in original code.
First, this is due to architecture used (very hard to test).
Second, routines have so many hardcoded things, I am amazed.
 
Look at the number shown in the attached Stream Parser screenshot. It represents the theoretical maximum speed the flash memory will be written. My Transcend class 10 card can do about 15MBs, and this value is somewhat lower than that. If my theory is correct then we would want to set the Overall Bitrate so this number equals just below the flash card maximum write speed. That way the camera will spend the minimum time possible servicing the flash memory write routine, and have processing resources available for other things.

Who knows, once that has been set to the maximum, the other settings may be able to be re-thought. It would be interesting to set this with a SanDisk class 10 card so that the transfer rate comes to 160Mbs (equivalent to 20MBs) - maybe even higher, and then see how high the Video Bitrate can go before instability appears. My thinking now is that the Overall bitrate is related mostly to card write speed.

Chris
 

Attachments

  • Burst.jpg
    Burst.jpg
    60.3 KB · Views: 0
The recorded bitrates should be about the same because they are governed by Video Bitrate - the only difference is that it doesn't fail with Native 24p checked (I think because it permits a faster instantaneous bitrate) - and the streams look pretty good.

Chris
 
Oh, it all begins to make sense now! If your theory is correct, this explains why AVCHD codec standard is limited to 24 Mbs. If this really requires double the overall bit rate for 24p frames, then this gives a bit rate of 48 Mbs. This is equivalent to 6 MBs, which happens to be the maximum write speed of a class 6 SD card.

So, if you use a class 10 card, then you can safely increase overall bit rate to 80 Mbs. This would allow 24p frames at about 40 Mbs bit rate.

Faster cards like Sandisk Extreme may be able to sustain 20 MBs (short of their advertised 30 MBs). So, maybe these cards can be pushed to 160 Mbs for a 24p bit rate of 80 Mbs.

I may have this all wrong, but the pieces do seem to fit together with your testing and known specs for AVCHD codec.
 
Vitaliy,

I've been looking at the code and the 18000000 constant appears in several places. In about half of them it starts with a value of 9000000, does a compare, and then changes to 18000000 depending on the compare result. The routines look fairly similar. Any idea what these routines are doing?

Chris
 
Well, it doesn't really need double the bitrate, at least not all the time - the average bitrate shouldn't change much. However, if the camera is buffering entire frames before writing, the Native frames will be twice as big, which could need a faster instantaneous write capability even if the average bitrate is the same. Also, I don't know how smart the flash write routine is; maybe it waits around assuming the card is slow and allows buffer overruns, etc...

One thing I'm fairly certain about - the calculated maximum bitrate limits the speed the camera can theoretically write to flash. It seems to me that the Overall Bitrate should be set to produce a maximum bitrate that matches your memory card.

Chris
 
Sorry, but I saw write-speed limitation failures in FHD mode on a Class 10 Sandisk Extreme 16GB card. I used the 40Mbps High Reliability Patch, modified with Native 24p at 86000000 Overall Bitrate:

native24p86moverallbitr.jpg
native24p86moverallbitr.jpg

native24p86moverallbitr.jpg
 
Last edited:
Did you get the failure with Navive 24p checked? I've been testing it all day and I can't get it to fail. Before I could reliably get a failure with Native 24p checked within 10 seconds. What was your test subject? I have to admit I haven't done any testing with Native 24p unchecked so I have no idea about how that might work out. I did, however, test all other modes (SH, H, L, and MJPEG) and got no failures. Stream Parser shows a maximum bitrate of 110,297,900 bps in my tests. Does that match what you got?

I'm not sure the empty frames aren't there for a different reason. They would be hidden with a pulldown. The only way to check is to take a wrapped clip, do a reverse telecine, and step frame by frame to see if there are any where the image doesn't change.

The Overall Bitrate constant exists several places in the firmware, so I suspect a little more research is needed.

I have to admit I'm somewhat confounded - after all, your card is faster than mine.

Chris
 
For example, here are empty frames in a SH mode clip - obviously Native checked has nothing to do with them.

Chris
 

Attachments

  • Empty720.jpg
    Empty720.jpg
    29.2 KB · Views: 0
You must log in or register to reply here.
Back
Top