Possible breakthrough on Native 24p!

slyn4ice said:
Just tested donwang's settings and it crashes immediately no matter what scene I have. 720 just crashes while 1080p freezes the whole cam. Something is fishy here. I suggest if we do test we should give all the information necessary to replicate the experiment - setting file for ptools, camera settings, testing procedure. All of this is pointless otherwise.
Click to expand...

+1. It would help a lot if people could give a comprehensive account of their stable 86Mbit settings so the rest of us can start testing it.
 
Vitaliy Kiselev said:
Guys, I want to warn you that this is thread in development section.
So, if you want to start testing and reporting here, better make new thread in general hack section.
Click to expand...

Got it. I will get a thread going tomorrow in the general section (if nobody hasn't done it yet - wink at OP) as it's night here. No settings will be posted here.

Good night guys.
 
Thanks Lpowell, I'll try some totally static shots with one of my primes. Actually, my test is very similar except that I use a fast shutter (1/500). If there is a breeze outside I just shoot static, if not I pan very, very slowly in order to keep the P frames busy. I've found that shooting with a fast shutter is much harder on the codec than 1/50th.

We may have a ways to go but I think there is something here. When I raised the Overall Bitrate I was unable to get the write speed error with my tests - otherwise I can get failures within seconds, so at least something is getting better. Well, I do know that more of fast cards' bandwidth is being taken advantage of at least.

Chris
 
I know in my enthusiasm I suggested trying other settings. But I agree with others that there needs to be some control here. I want to stick with the Lpowell stable 40Mbs settings for now and drill down the Native 24p issue with just those settings first. His settings have proven (at least, to me) to be utterly reliable and I want to eliminate other stability issues and just concentrate on the Native 24p issue.

Today I plan to first try to replicate Lpowell's failure scenario, and then try to determine if setting the Overall Bitrate higher actually undermines stability with Native 24p unchecked as well.

Chris
 
I tried some totally static, very high detailed scenes and I got a failure with this new patch. Rats - I was hoping this would work better.

There is a definite improvement - but the reliability is not there. It looks like I'll have to do some more digging.

Vitaliy, when the Overall Bitrate patch is set which instance of that constant is changed? Just the one in the initial setup area?

Chris
 
Vitaliy Kiselev said:
All of them, as far as I remember.
Click to expand...

Hmmm... now I'm wondering if the routines that start at 9000000 and go to 18000000 based on a conditional need to have the 9000000 changed to 1/2 the Overall Bitrate patch value. What do you think?

Is that what Overall Bitrate 2 does?

Chris
 
Last edited:
I tested the Lpowell settings with the Overall Bitrate raised, but Native left unchecked and I got no crashes. It seems that raising the overall bitrate does not undermine stability by itself.

I also tested the factory default firmware for empty frames in 720p mode. On a static, high detail scene there are quite a few empty frames. It looks like there is some sort of threshold value for changes in the scene that must be exceeded for video data to be produced. So if you have a static scene with something small changing - like a bug moving around - the codec will miss it sometimes. A majority of empty frames (about 60%) occur after I frames. It seems unfortunate to me that there is a threshold as there is little to be gained by throttling what little data would have been produced without it.

When I look for empty frames in 720p mode with the Lpowell settings most of the empty frames occur after every second I frame. This is a strange pattern because if it were simply because of the I frames it seems to me that they would appear after every I frame, not after every second one.

Chris
 
It works for me. :cheesy:
stress test in FHD mode with foliage (sorry no grass here)
sd card panasonic class6 4Go and class 4 16Go
pal=>ntsc checked
native 24 checked
video bitrate: 50 000 000
overall bitrate: 86 000 000

MTS files arounds 44000 Kbits/s (this bitrate used to crash before with my class 4 sd card)
framerate 23.976
 
cbrandin said:
I've been digging into the GH1 firmware code and found that overall bitrate appears to have a big effect on the maximum speed that flash memory writes can occur. I’ve always wondered why the write speed error comes up in AVCHD mode when clearly the data rates are well below what the card can handle. I think I found the answer.

When the write speed error comes up it’s because the calculated maximum write rate has been exceeded – whether the card can handle more or not. I tried the Lpowell settings but changed two things: I doubled the overall bitrate to 86,000,000 and I checked Native 24p. Upping the overall bitrate nearly doubled the calculated flash write rate (i.e. the fastest write rate the camera attempts to write to flash). It appears to be working! I’m not getting any lockups and the clips look great. I tested this with highly detailed scenes, fast shutter speeds, everything I normally do to break the codec and it seems to be standing up.

We probably need to do some fine tuning and more testing (hint, hint, Lpowell), but so far it’s looking pretty good. I ran my tests with a Transcend class 10 card.

Chris
Click to expand...
Any further progress on this? And why does Native 24p have to be checked, does it affect the data rate as well? So there would be a scenario in which Native 24p would be *more* stable?
 
cbrandin said:
I tested the Lpowell settings with the Overall Bitrate raised, but Native left unchecked and I got no crashes. It seems that raising the overall bitrate does not undermine stability by itself.

I also tested the factory default firmware for empty frames in 720p mode. On a static, high detail scene there are quite a few empty frames. It looks like there is some sort of threshold value for changes in the scene that must be exceeded for video data to be produced. So if you have a static scene with something small changing - like a bug moving around - the codec will miss it sometimes.
Click to expand...
I've seen this behavior in my 50Mbps AVCHD Fast Action Patch as well. While we don't yet have a solution for a reliable Native 24p video mode, your research has enabled me to increase this patch's peak bitrate in FHD and SH videos up to 55Mbps:

http://dvxuser.com/V6/showthread.php?t=222791

I'm also puzzled by the sporatic occurance of empty frames in the middle of an otherwise data-packed 15-frame FHD GOP. It seems extremely unlikely that a single frame in a full-motion video would happen to be virtually identical to the previous frame. Perhaps the GH1 can sometimes fail to complete its sensor scan before encoding of the next frame starts? In that case, the camera would be forced to reuse the data from the previous frame, resulting in an empty P-frame (since nothing has changed). This could be an intentional failsafe mechanism to give the CPU time to catch up when it has fallen behind the framerate.
 
It might be some kind of bitrate failsafe. I don't think so, though, and I hope not. It looks like more of a buffer failsafe. I'm trying to analyze the encoder routines. Hopefully there is something in there that assumes that 24p footage will be wrapped and an output buffer is allocated assuming half-frames instead of full ones. Who knows at this point - but I'm looking!

Chris
 
cbrandin said:
It might be some kind of bitrate failsafe. I don't think so, though, and I hope not. It looks like more of a buffer failsafe. I'm trying to analyze the encoder routines. Hopefully there is something in there that assumes that 24p footage will be wrapped and an output buffer is allocated assuming half-frames instead of full ones. Who knows at this point - but I'm looking!

Chris
Click to expand...

Chris, just curious, with this high bit AVCHD settings you've been working on for 24PN. Is it any more stable using 24 over 60i wrapper rather than selecting 24PN?

If your able to get the bit rate at 80mbits+ for avchd; 24PN or 24PSF, that would be nice if it's stable with either one.

Pappas
 
PappasArts said:
...Is it any more stable using 24 over 60i wrapper rather than selecting 24PN?

If your able to get the bit rate at 80mbits+ for avchd; 24PN or 24PSF, that would be nice if it's stable with either one.
Click to expand...
I did a week's worth of testing on 1080p/60i peak bitrate settings: 55Mbps, 77Mbps, and 110Mbps (as reported by GH13 Stream Parser). The clues I picked up from Chris' research led to a reliable 50Mbps AVCHD patch for Class 6 and higher SD cards:

http://dvxuser.com/V6/showthread.php?t=222791

I can get 77Mbps to produce stable 720p30 AVCHD videos, but not with a 3-frame GOP. At this point, I think 55Mbps is the highest peak bitrate the GH1 can handle at 1080p.
 
Last edited:
The Native mode still has issues - some bugs, I think. Theoretically Native mode should be able to achieve higher bitrates than wrapped 60i mode because it has no duplicate frames created by the telecine conversion that have to be written to flash. There's also the issue of color subsampling - it's a little different in true progressive mode (most like it better) and concerns edges on highly saturated colors. The Native mode is worth fixing because between the higher potential bitrate devoted to unique image data, and the better subsampling, and not having to reverse-telecine clips to edit them; there is quite a bit to be gained.

Alas, you'll have to wait until these bugs can be resolved.

Chris
 
Last edited:
Actually, I might have it wrong. We may not be screwed after all - stay tuned. Many thanks to DaLiv for pointing out my error! I misread an address and went to the wrong places.

As for my day job, I do high tech forensics (processor architecture, encryption, hardware and software analysis, etc...) and testify at trials, etc... A big part of what I do is basically hacking. Also I'm chief scientist for Xpriori (nee NeoCore) - semi retired.

Chris
 
cbrandin said:
As for my day job, I do high tech forensics (processor architecture, encryption, hardware and software analysis, etc...) and testify at trials, etc... A big part of what I do is basically hacking. Also I'm chief scientist for Xpriori (nee NeoCore) - semi retired.

Chris
Click to expand...


So your job is finding childporn on peoples computers? I see why you have such a good background on cameras. Is it true that modern sensors include stenographic signatures that allow for a picture to be traced back to a particular camera?
 
You must log in or register to reply here.
Back
Top