Potential Clip Problems, Please check...

[Bassman2003]

Hello,

I filmed a wedding over the weekend and I am working with the footage and an issue has shown up.

(I will have my personal review of the camera forthcoming when I get some time).

On long clips, I am noticing a few frames dropped around the 26 or 27 minute mark or the 3.99Gb mark.

So it looks like the seemless transition between clips is not working!

I am using Panasonic Class "6" 32 gig cards

If I could ask, could owners please check if this happens with their setup?

Thanks.

I have another wedding to shoot this weekend and this is not comforting...

[Evro]

Hey Bassman I too would like a response from someone as this can't be a good thing - especially if it happens during vows or toasts... :(
I'm due to receive my camera at the end of the month.

[manglerBMX]

how are you handling the clips? are they getting transcoded to another format or are you working with an NLE that supports native avchd?

[Mark Whalen]

Also, what quality setting were you using when you encountered this problem? Is your issue reproducible at other settings as well?

[Everts]

So this camera is capable of producing drop frames afterall, hmmmm.
I wonder if this has anything to do with it.
Fat32 has 4 Gb filesize limit.
NTSF has none.
The SHDC cards are all FAT32 formatted .
I can imagine some sort of delay happening in the camera when creating a new clip while recording

I dont do weddings but I would really hate for this to happen if ever I have to shoot for that long. Come to think of it interviews can get messy. :(
Still waiting on my camera.

[Thomas Lew]

Not that you should have to find a workaround but couldn't you just stop recording and record again a second after during a down time when u don't need to continuously record?

[manglerBMX]

when shooting a wedding, or any event with multiple cameras, it makes it a million times easier to sync them up when they don't have breaks in recording.

[PKraft]

On long clips, I am noticing a few frames dropped around the 26 or 27 minute mark or the 3.99Gb mark.

Working on a mac?
Then it is a know problem if you are using OSX 10.5.5 and Quicktime 7.5. or older.
See: www.perian.org and www.versiontracker.com/dyn/moreinfo/macosx/30931

[ilauzirika]

I haven't had any problem with spanned clips so far, thay all played back perfectly.
clips were converted to dvcpro hd with the pany converter.

[Averdahl]

So it looks like the seemless transition between clips is not working!
- Are you editing native AVCHD clips?
- What editing application do you use?
- Have you tried to convert the clips to DVCpro HD with Panasonics transcoder?

I have "only" seen this issue with native AVCHD clips in Premiere Pro CS4. The problem went away when i transcoded the clips with Panasonics AVCHD to DVCPro HD converter.

/Roger

[Bassman2003]

I am using Canopus HQ, but the problem is with the source file.

When I look at the AVCHD file, the last 30 frames are the same.

It looks to me like the GOP structure was caught in an uncomfortable place when the 3.99Gb limit was reached.

So this is not a transcoding error, this is a recording error with the camera.

I was asking if anybody else could test this.

I was filming at the highest quality setting with 720p60 without pre-record on.

Thanks, and hopefully this is just a setting.

[jeff9329]

when shooting a wedding, or any event with multiple cameras, it makes it a million times easier to sync them up when they don't have breaks in recording.

You would stroke out editing weddings I shoot when each camera typically has about 150 clips each and the primary audio tracks are on digital audio recorders.

That is a potential real problem with the 4GB thing if it turns out to be true. It would certainly need a firmware fix.

My longest clips so far have been 17 to 20 minutes in length and I have seen no problems yet.

[Evro]

You would stroke out editing weddings I shoot when each camera typically has about 150 clips each and the primary audio tracks are on digital audio recorders.

Even worse when your digital recorders are Zoom H4/H2 series that encode with a slight time lag and you have to resync the audio by about 2 frames every couple of minutes to match the vision - would be nice if they told you this stuff beforehand :(

[Bassman2003]

I called Broadcast tech support and explained the severity of this issue and strongly asked for a call back as they have not called in the past, but nobody returned my call.

When I called back after some hours they said they were closed (4:30 central time)

I guess this camera does not rate or the message of a serious possible issue did not reach the target.

Oh well, I will try again.

[booggerg2]

Bass: Can you see these frame problems when playing back from the camera?

[Bassman2003]

I have not tried that.

[Bassman2003]

Tested last night with and without "Pre-Record" and the results were the same.

The last 30 or so frames repeat at the end of the 4gig clip, then motion returns with the start of the next clip.

Looks like an issue unless somebody else knows of a setting I can change for this?

Anybody else test for this?

Last night I just pointed the camera at the CNBC channel on T.V. for 35min as the stock ticker never stops.

[Averdahl]

Tested last night with and without "Pre-Record" and the results were the same.
Have you, for testing purposes, tried Panasonics AVCHD to DVCPor HD converter to convert two clips to see what happens?

/Roger

[Bassman2003]

No, I am looking at the source AVC clips directly.

[ajcourtney]

Well I guess I'll be the first to say this problem renders this cam completely useless for any serious work. I must also say that it does not surprise me one bit since those of us who have been fooling around with AVCHD consumer cam's for the past 10 months have encountered the 2GB (FAT) hard file limit with every AVCHD cam created to date. Some manufacturers (Sony) have written software that will seamlessly resplice these files back to the original contiguous clip, while others (Canon) continue to ignore the issue altogether and hope that a third party ISV (Pixela) will eventually fix it for them (not happening).

The fact that this issue continues to resurface even with today's advanced filesystems proves that AVCHD clearly cannot be taken seriously. I firmly believe that the consortium is either directly or indirectly responsible for this extreme shortsightedness and the irresponsibility of the committee driving this format is entirely to blame for the complete hit-or-miss extended contiguous recording capabilities that we are currently witnessing in these consumer electronics manufacturers' products.

My advice to you Bassman2003 is to return the camera as defective and demand a full refund. Furthermore, anyone who has pending orders should think about cancelling those orders until Panasonic steps up to the plate and proves it has delivered a permanent fix.

I spent four months earlier this year going round and round with Canon support (idiots) and had to PROVE to them there was a replicable problem with every HG-10 AVCHD consumer cam. In the end, they denied accountability for this issue and instead defaulted to blaming the accommodating software package as being unable to correctly re-splice the <2GB clips back to the original contiguous file as displayed on the cam's LCD.

Actually, this is sounding like a far more serious problem because the Panny is apparently corrupting the video stream - something that not even the cheap HG10 ever did. At least the little consumer cam left the video and audio well enough alone. And we could use the copy /b DOS command and manually re-splice the files back together into the original contiguous clip with ZERO dropped audio samples or vid frames.


I am so sick and tired of these stupid electronics manufacturers pawning off defective crap products and then trying to absolve themselves of any responsibility for these defects. It's not until the market starts DEMANDING a quality product and voting with its collective $$$ that we will see the status quo change. Similarly, AVCHD has proven to be a complete joke to date. What kind of an idiot imposes filesystem limitations that are part of legacy systems not even in use with today's video editing PC or Mac? Are you kidding me? Do you seriously believe some moron is going to use Win'98 to edit AVCHD footage? GET REAL.

One of the strongest features of AVCHD is the possibility of extended record times during acquisition vis-a-vis traditional tape-based HDV. So far, that capability has yet to materialize without resorting to using special software designed to overcome brickwall limits in the filesystems used by the AVCHD products. Truly pathetic.

HEY CE MANUFACTURERS - GET A F'ING CLUE ALREADY.

[ltf3]

Problems here too...

Seems our clips (from two cameras shooting 720p 24fps) don't transfer into FCP 6.02 correctly. All clips break up (blocks) at the very end (1-2 secs) but more severely, when the clips are transfered into FCP there is always material missing at the end!

A 1 min clip has 1-2 secs just missing! A 15 minute take has over 30 secs missing!!!!! The "record extra material" work around doesn't cut it in that case!

The clips play OK in the FCP Log and Transfer Viewer (leaving out the break up at the end) but there's no way to explain the missing footage.

We are transferring data using a USB card reader ... we'll try using the camera as the card reader when they get back in.

Bit of a worry though!

Lee

[Evro]

For people that shoot weddings every weekend, transcoding to DVCPro HD seems like a very bulky & time consuming task - every wedding will need 500GB of HDD space instead of 70GB, plus time to transcode 5-8 hours of footage. The whole idea behind going solid state was to save time by skipping the tape capture step yet now we have to spend that time waiting for transcode operations. At least when you were capturing from tape you were also viewing and making notes - transcoding time seems completely unproductive!

My camera arrives sometime next week and I'm not feeling good about it after reading this - hmmm... maybe I should have gone with the EX1 after all - the issues with that camera are relatively minor compared to what I'm reading here! The HMC-150 can have the world's greatest low light abilities but if I loose one minute of footage for every 30 minutes I shoot I can get into serious trouble.

Can someone from Panasonic respond to these file corruption problems? Surely Panasonic would have tested & seen these issues before releasing the camera for distribution. Some of us rely on making our living & supporting our families through video production! I can live with the odd tape drop out now and then but I could NEVER forgive Panasonic if this camera's footage caused me the problems people are talking about here.

This whole thing reminds me of when Microsoft rushed the buggy release of Windows 95 to meet marketing demands, and to be followed by a string of service packs for the next year or so to patch the bugs.

[impressive creations]

Yeah, I should get my cam tomorrow and like Evro, I do weddings full time. This is what provides for my family. This will be the first thing I test out. If it messed up. It's going back. I am willing to wait for AVCHD to be supported natively in FCP, but if it's dropping frames and you can see it in your view finder. Ouch... I want this camera so bad.. but seriously, this is a major issue. I hope its just bassmans camera and not all of them. Itf3, are you having the same problem if you just watch your footage through the cameras view finder.

Thanks..

[Bassman2003]

Update:

The clips play back seemlessly in camera.

So that rules out the camera recording malfunctioning, which is a very good thing.

The problem is how to get the spanned clips into my editor (Edius).

Every way I have tried brings the clips in as separate files and I think Edius does not know how to glue them back together and this messes with the GOP structure.

I am in need of suggestions as I have run out of things to try from what I know of.

The AVC viewer from panasonic is kind of clunky and ends up with the same information in the stream folder that I can get from a straight transfer from the SD card.

So, the camera is great, just need the know-how to properly work with the clips...

[gmoe]

Hey Bassman-

I had a tiny issue with this as well and I'm using Edius 5.

I did exactly what you did in terms of checking the clip in camera and it plays perfectly but on one of my canopus HQ clips I had trouble encoding. I had the same problem when transcoding to DVCPro. I was in a rush and didn't need that clip anyway so it didn't occur to me that this could be a serious problem until I read this thread.

I think Edius and Panasonic need to create an update on their transcoding software. :(

This is not good news.

-G

[ajcourtney]

I am in need of suggestions as I have run out of things to try from what I know of.

at least the camera works. if these are straight .mts files, open a command prompt and try to the copy /b command to join the files together (binary). It works for the Canon. The problem obviously then becomes the "housekeeping" required in order to manually join the right files in the correct order. Definitely not ideal, but might work here...

[Bassman2003]

I wish there was a way to bring the files in as entire clips so a spanned clip would be one long non-stop file for the editor.

There so many folders in the SD card structure, seems strange that what gets copied into the editor is just the stream folder contents.

This is probably the problem.

[impressive creations]

Whewwww..That was a close one.. My camera shows up tomorrow.. Thanks for checking that out Bassman.. I'm sure things will get ironed out here soon..

[ilauzirika]

I'm trying the camera at this moment, I'll let it film my hands for half an hour and see what happens to the sppaned files.

[Everts]

Getting nervous here ...
Im about to place an order
Does David Jimerson (http://www.dvxuser.com/V6/member.php?find=lastposter&t=150613) know if this problem exists with Premiere CS4 seeing Bassman said its not the camera but the convertersoftware ?
Thanks .

[jeff9329]

Im having no camera problems/bugs and no editing problems at all in Vegas 8.0c.

It's a great camera.

[Averdahl]

...if this problem exists with Premiere CS4
Yes, i see the same issue with native AVCHD footage in PPro CS4.

If i convert the footage to DVCPro HD or capture via HDMI into a Intensity card the problem is totally gone. It takes more time but it is workaround that i will use since it works and solves the problem.

I have seen the very same issue with a Canon AVCHD camera, so it not the HMC-150 that is faulty.

/Roger

[Everts]

Thanks Roger.

[ltf3]

Hi

More Info :

If you Log and Transfer 720p 24fps footage from a card reader *or* the Camera into FCP 6.02, transcoding to Apple Intermediate Codec, a certain amount of material is chopped off of each clip. The longer the clip, the more is lost (13 mins loses 35 secs!)

We are now using Toast 9 in its Convert mode to transcode to Quicktime, AIC, 24p, and it does it perfectly.

Then import into FCP.

No reason to expect other footage frame rates/resolutions to be any different. My bet is an FCP issue.

We'll forward to Apple feedback...

Lee

[BobDiaz]

Well I guess I'll be the first to say this problem renders this cam completely useless for any serious work. I must also say that it does not surprise me one bit since those of us who have been fooling around with AVCHD consumer cam's for the past 10 months have encountered the 2GB (FAT) hard file limit with every AVCHD cam created to date. Some manufacturers (Sony) have written software that will seamlessly resplice these files back to the original contiguous clip, while others (Canon) continue to ignore the issue altogether and hope that a third party ISV (Pixela) will eventually fix it for them (not happening).

The fact that this issue continues to resurface even with today's advanced filesystems proves that AVCHD clearly cannot be taken seriously. I firmly believe that the consortium is either directly or indirectly responsible for this extreme shortsightedness and the irresponsibility of the committee driving this format is entirely to blame for the complete hit-or-miss extended contiguous recording capabilities that we are currently witnessing in these consumer electronics manufacturers' products.

....
[/rant]

What we are seeing here is a software problem, not a hardware problem.

Depending on the software used, some video may be lost, resulting in the glitch between spanned clips. For now, work a rounds are required, but I would hope that Panasonic is working with the software companies to help fix the the glitch in the conversion.


Bob Diaz

[ajcourtney]

I wish there was a way to bring the files in as entire clips so a spanned clip would be one long non-stop file for the editor.

There so many folders in the SD card structure, seems strange that what gets copied into the editor is just the stream folder contents.

This is probably the problem.
So, am I correct in the saying the cam does not include software designed to re-splice the clips back into the original contiguous clip upon transfer to a PC/Mac?

[ajcourtney]

What we are seeing here is a software problem, not a hardware problem.

Depending on the software used, some video may be lost, resulting in the glitch between spanned clips. For now, work a rounds are required, but I would hope that Panasonic is working with the software companies to help fix the the glitch in the conversion.


Bob Diaz
I understand Bob because those of us who have been using the AVCHD consumer cams for the past 10 months have already dealt with this issue. My point is it is Panasonic's responsibility to provide software with the cam to correctly re-splice the various <4GB clips back to the original contiguous clip perfectly upon transfer to the editing computer. If they cannot do this, then THEY are the problem since THEY have decided to impose an outdated filesystem restriction upon the user that is completely irrelevant in today's world.

[Jim Simon]

/Roger

Hey there. Didn't know you lurked over here. Haven't seen you much over at the CS forums.

[Jim Simon]

HEY CE MANUFACTURERS - GET A F'ING CLUE ALREADY.The problem is actually with Mac users. They can't use NTFS, so solid state cards intentionally screw the majority (PC users) by formatting their drives with FAT32 to accommodate the minority (Mac users).

A very simple solution would be for solid state card makers to allow us to format the cards as we see fit (or drives in the case of Firestore). With NTFS (or HFS for Macs), long record runs would be a single file - problem solved.

Lacking that, different cards for PC and Mac users should be made available.

Bottom line is, all solid state devices/external drives should at a minimum allow full functionality for the vast majority, meaning NTFS formatting. If they still want to accommodate Mac users, fine. Just don't screw the majority in the process.

(OK, that's off my chest. I'm better now.)

[Justyn]

Jim, that's so far off from the actual truth. Macintosh users are actually more of a majority in this business that PC users. 60 percent of all HVX users were on a Mac and I'm sure that the trend will follow suit with the 150 and such. PC's are not the majority in the creative arts and far from it... certainly in terms of the working and successful people that I know. Maybe 1 in 20 people I know use a PC.. and God bless them for taking that route... so I suggest you might even want to conduct a poll, but I'm sure you'll get some results that confirm what I'm saying.. and at the least, it will be 50/50 and maybe more and more are going with the better value in this 150.


cheers.

[PKraft]

The problem is actually with Mac users. They can't use NTFS, so solid state cards intentionally screw the majority (PC users) by formatting their drives with FAT32 to accommodate the minority (Mac users).Who says that a Mac cannot handle (read from/writeto) NTFS? It is better to get informed before to start bashing.


A very simple solution would be for solid state card makers to allow us to format the cards as we see fit (or drives in the case of Firestore). With NTFS (or HFS for Macs), long record runs would be a single file - problem solved.I absolutely agree as for NTFS, HFS not necessary, see above.


Lacking that, different cards for PC and Mac users should be made available. Not necessary, see above.


Bottom line is, all solid state devices/external drives should at a minimum allow full functionality for the vast majority, meaning NTFS formatting. If they still want to accommodate Mac users, fine. Just don't screw the majority in the process. I absolutely agree as for NTFS, with a bottleneck of FAT 32 due to PC users though. FAT32 being the least common denominator of all platforms to be included.


(OK, that's off my chest. I'm better now.) OK, that's off my chest. I'm better now. ;-)

[ajcourtney]

Well it would be nice if the hosting device gave the user the opportunity to format the card in whatever filesystem he wanted - probably impractical - but that is certainly not what is happening today. So, my previous question becomes all important:

Does Panasonic provide software to seamlessly reassemble these <4GB clips back into the original contiguous clip as displayed in cam? If the answer is "no" then I believe, based upon my past 10 months' of experience dealing with this AVCHD file size limit, the user will be entirely reliant upon his video editing software of choice to perform this function for him. And again, based upon past experience, you may be waiting for a very, very long time for this issue to be addressed, IF IT EVER gets addressed by your video editing software vendor.

jeff9329 writes: "Im having no camera problems/bugs and no editing problems at all in Vegas 8.0c."

Are you sure about that? Because I doubt that Vegas will seamlessly reassemble these clips back to the original file because it can't do it for the Canon cam's. So if the Panny files are a straight binary join like the Canons', I would expect to see 1-2 dropped frames and some dropped audio samples. The easiest way to test the audio would be to pipe in a live concert feed, set the cam on the table, press record, and walk away - which is exactly what I did with the Canon. When I brought the multiple files into Vegas (build 217), every splice point exhibited dropped audio - the rhythmic nature of continuous, live rock made it extremely easy to detect these splice points. So perhaps you could run a similar test and let us know what happens.

In any event, I would imagine Panasonic to come back and say use the DVCPro HD converter if you don't want any problems. Other than that, complain to your video editing software manufacturer because it's their problem, not ours. And that means you now have a mandatory conversion in order to avoid the issue, at least at this point in time.

[Barry_Green]

The AVCHD file system works. If there are problems, they're with the NLEs, and there's nothing Panasonic could or would do about that. The camera is hardware. NLEs are software. NLEs are easily fixable. Cameras not so easily fixable, and even if they were, there's not a problem.

You can play spanned AVCHD files back in the camera just fine; I've got several examples of spanned AVCHD files and they play great. You can play them back in the AVCCAM Viewer software, you can play them back in a blu-ray player, they work fine -- there is no stuttering or glitching or anything. You can play them back in a PS3, they work fine.

Any issues you folks are encountering are with the NLEs. It's easy to demonstrate that the 4GB files and the spanning structure work flawlessly.


Does Panasonic provide software to seamlessly reassemble these <4GB clips back into the original contiguous clip as displayed in cam?
Not sure what you're asking. I assume that by "<4GB" that you actually meant ">4GB", right? Because any clip less than 4GB should already be one contiguous file.

I think what you're asking here is: does Panasonic provide some software that will take the chunks of clips caused by the FAT32 file format's 4GB file size limitation, and export these spanned clips into NTFS or OSX or some other operating system's file format as a contiguous file that's greater than 4GB, right? So 10GB of source clips (being two 4GB clips and one 2GB clip on FAT32) would become a 10GB clip on NTFS, right? If that's the question being asked, then no, there's no software that does that as far as I know. Seems like a trivial piece of software to write up though; I'm sure some 3rd party could do it in about half an hour. Maybe ShotPut Pro could do it (if not in the current version, in a simple revision?)


In any event, I would imagine Panasonic to come back and say use the DVCPro HD converter if you don't want any problems.
Well, but see, that's kind of funny because the DVCPRO-HD converter uses the same 4GB file size limitation. So you'd end up with clips that were actually 4x as spanned (assuming 1080 clips). If you had 6GB of source footage, that's one spanned splice in your original AVCHD footage, but you'd have about 25GB of destination footage, making for six spanned splices of DVCPRO-HD footage. Yet we know that all NLEs handle spanned DVCPRO-HD just fine.

It's not the spanning that's a problem. That's a demonstrated and proven system. If there's an issue it's with the way the individual NLEs are handling it. Which means that yes, you'll need to get an update from your NLE manufacturer.

[Bassman2003]

Your correct.

I just spoke to tech support and the tech said he would look into it.

When I mentioned that Panasonic should release a little utility that you could pass the clips through to "glue" them back together, he crawfished a bit and did not want to go there.

From a consumer point of view, and this being called a pro cam, they better address this because I purchased this camera to be able to record for long periods of time!

I also would rather the material stay full raster and not have to go to DVCPro HD.

We will find out!

[ajcourtney]

AVCHD is not a filesystem. It's a format that uses outdated filesystems that have created this mess of incompatibilities that people have experienced, are experiencing, and will continue to experience. AVCHD's compression algorithm may be superior to MPEG-2, but its implementation thusfar has been a complete crapshoot for the consumer WRT these software inconsistencies. The ONLY vendor who has been able to sidestep this file size problem has been Sony because it provides the PMB (or whatever the acronym is) software designed to re-splice these <2GB (FAT) and <4GB (FAT32) clips back into the larger contiguous original "take" (defined as the moment in time record was pressed first to initiate recording and a second time to end the recording) upon transfer to the editing computer.

If the software is so easy to create, as you put it, tell that to Pixela - the software company Canon used to outsource this very task. They have chosen to impose their own limit of a one hour contiguous clip last time I spoke with their development team. The software must be smart enough to recognize that clips 1,2,3-5 belong to the same contiguous file; yet clips 6-9 are individual takes <2GB or <4GB (depending on FS). Unless the cam is providing some sort of metadata in the broken clips from the original contiguous take, how is the software smart enough to know which clip is part of a larger contiguous clip, and which is a separate take?

Trust me, this is a very complicated issue that has been discussed at great lengths since the first AVCHD cam's were released. Many of us have seen firsthand the games these manufacturers have played this year to absolve themselves of the responsibility to get the clip as it appears on cam into your editing computer PERFECTLY. You say it isn't their responsibility. I disagree and say it is.

This will prove to remain the single most important AVCHD topic in the nearterm for anyone requiring anything more than ~20 minutes or so of continuous footage.

[Barry_Green]

I don't know what you've been dealing with, but the 150 was designed to handle recording 12 HOURS of continuous recording, and it does it quite well.

And yes, there is metadata in the clips that specifically tells the computer how to stitch together HMC150 clips. I would assume, but don't know, that the HSC1U and HMC70 both have that capability as well.

I don't know what Canon or Pixela have done. If Canon's restricting themselves to an hour, that's their choice, but the HMC150 has no such restriction.

The file format (FAT32) was chosen because it's universally compatible. File formats that support files larger than 4GB are not universally compatible. FAT32 was the only reasonable choice.

As far as software to re-stitch it, as a retired professional computer programmer I do feel on rather safe ground to say that, at least as far as HMC150 footage goes, it would take someone about half an hour to write an app that stitched the files back together into a seamless, lossless 12-hour contiguous take of up to 32GB.

Heck, I wonder if the AVCCAM Writer software already does that? Maybe there's an option for it? I should try to dig into it.


You say it isn't their responsibility. I disagree and say it is.

It is their responsibility to provide a system that works. They have done that. You can play back 12-hour clips seamlessly. It is the NLE company's responsibility to make sure that their NLE can work with the footage as intended. But all the hardware already works exactly as intended.

[ajcourtney]

If the camera was the sole object as a means to an end, then yes, the system performs flawlessly. And that will be Panasonic's response - guaranteed. If, however, you're like 99.99999% of users, you will require the camera to interface with your chosen NLE. And this is the point of failure. Your opinion is it is the NLE's responsibility to correctly re-splice these broken clips back into the original contiguous file. It's my opinion that it's the camera manufacturer's responsibility to perform that operation since that party is the one that decided to use a filesystem that required the files to be divided up in the first place. Neither one of us is right or wrong. It's an opinion.

As it stands today, I will vote with my wallet by choosing not to purchase this cam until its AVCHD footage is universally supported by all NLE's advertising native AVCHD editing capabilities. How that feat is accomplished is largely irrelevant; but I would definitely prefer to see a solution provided by Panasonic since that would remove the mandate from the ISV's for supporting this product in future iterations of their software packages - something that few, if any, ISV's would ever commit to indefinitely.

BTW, as was already mentioned, Mac's running OS X and later (and possibly even later versions of OS 9) can read NTFS formatted partitions. And I believe some UNIX variants can do so as well. Not being able to write to that partition is irrelevant in this workflow model.

[Bassman2003]

As far as software to re-stitch it, as a retired professional computer programmer I do feel on rather safe ground to say that, at least as far as HMC150 footage goes, it would take someone about half an hour to write an app that stitched the files back together into a seamless, lossless 12-hour contiguous take of up to 32GB.


Hey Barry,

I am sure this will get resolved, but if it easy to write this kind of software, why doesn't Panasonic put out a small program like this for its customers?

We have edit what we shoot!

Is the NLE specific or all of the NLEs?

I know Edius is not spanning the clips, native or with CanopusHQ.

[jeff9329]

jeff9329 writes: "Im having no camera problems/bugs and no editing problems at all in Vegas 8.0c."

Are you sure about that? Because I doubt that Vegas will seamlessly reassemble these clips back to the original file because it can't do it for the Canon cam's. So if the Panny files are a straight binary join like the Canons', I would expect to see 1-2 dropped frames and some dropped audio samples. The easiest way to test the audio would be to pipe in a live concert feed, set the cam on the table, press record, and walk away - which is exactly what I did with the Canon. When I brought the multiple files into Vegas (build 217), every splice point exhibited dropped audio - the rhythmic nature of continuous, live rock made it extremely easy to detect these splice points. So perhaps you could run a similar test and let us know what happens.


AJ:

I am currently editing a long wedding project. It's a multicam (only 1 HMC150) edit and I have not rendered any video yet. When I let the only 4GB clip in this project run to the next clip in the NLE, I don't see a stutter. That's not a good test like a render will be though. I only have one of these long clips as I never just let the cameras run continuously. We shouldn't have to work around issues like this, but for my style of shooting, I almost never have any single clip of much length before a camera angle switch, so it won't affect me. As Barry said in a previous post, maybe some Pana software actually uses all that header info and reassembles the clips. I have not needed any software other than NLE, so I never even looked at the software provided with the camera.

Another note on the HMC150 editing; I ran the deshaker script on a bunch of HMC150 handheld clips and it worked flawlessly and the footage looked pretty lossless. Clip renders using the script were giving me about 4 FPS which is livable.

Jeff

[mcsmooth]

I don't remember the AVCCAM viewer stitching the files (not 100% on that), but as mentioned, there is no need for it to since the clip information references it properly as one clip. Any software (viewer, editor, etc) that reads the format properly will also show it as one clip, so it really should have no impact unless you are trying to copy stream files by themselves (not the best idea for most purposes).

Agreed that FAT32 is a necessary choice for right now since it works with everything. It makes sense for programs NOT to automatically stitch the files in case you wanted to move the them back to the card and stay within AVCHD's specs. I can see people wanting that feature to be able to convert to a single stream file though... If programs do not already do this, expect tools to come about soon since this has been done with other forms of mpeg for a long time and does not require much work. Anything that smart renders would be able to do this too.

Look at most any movie's file structure on DVD and you'll see the streams are split into 1GB files for any video bigger than a 1GB (which most movies will have for the feature). This is a similar limitation that works the same way... every player (or software) that follows standards will see and play these back as one clip without skipping a frame.

[mcsmooth]

Hey Barry,

I am sure this will get resolved, but if it easy to write this kind of software, why doesn't Panasonic put out a small program like this for its customers?

We have edit what we shoot!

Is the NLE specific or all of the NLEs?

I know Edius is not spanning the clips, native or with CanopusHQ.

Not just to you... but to all that are waiting on Panasonic, sure it would be a nice gesture, but I wouldn't hold your breath. Why should they be required to fix a problem with someone else's software? This IS NLE specific! The camera follows the AVCHD spec, so it is up to the software to follow suit. There are many posts in here of other software that does not have this problem.

On that note, I love this camera, but I would not recommend it (or ANY other camera with a new codec) to someone that has important/immediate editing to do and has not tested the workflow first. You are buying into something that has not been proven yet and will require some sacrifices until compatible software/hardware catches up.

[ajcourtney]

When I let the only 4GB clip in this project run to the next clip in the NLE, I don't see a stutter.

It's likely you wouldn't see a problem at the splice point. It's better to dump the audio into SoundForge and evaluate the wave file visually. When I did that with the Canon footage, I was able to count a specific number of lost samples (@ 48K/16-bit) that created a very specific audio "hiccup" at the joining point between multiple broken clips. Again, the rhythmic nature of rock music (in this case a live Collective Soul show) made every splice point totally obvious on the timeline in Vegas.

These were raw .mts files opened directly onto the Vegas timeline

[ajcourtney]

Not just to you... but to all that are waiting on Panasonic, sure it would be a nice gesture, but I wouldn't hold your breath. Why should they be required to fix a problem with someone else's software? This IS NLE specific! The camera follows the AVCHD spec, so it is up to the software to follow suit.

The file size restrictions imposed by the decision to use specific filesystems has nothing to do with the AVCHD committee. I contacted the consortium on this very point and did not receive confirmation that the AVCHD spec includes a requirement to (1) use filesystem "x" which would impose file size limits on video footage and (2) incorporate a software utility designed to sidestep this file size restriction and correctly reassemble camera clips that exceed any file size limit imposed by the particularly file system incorporated in a vendor's hardware product. Actually, they make it pretty clear that it is up to the particular vendor to decide how to logistically implement the AVCHD format, which, IMO, is one of the primary reasons we are left to deal with this problem.

Therefore, when you say it is the responsibility of the NLE vendors to follow suit (I assume you are implying that these NLE's are NOT AVCHD "spec-compliant"), your statement makes no sense because these NLE's are not "out of compliance." And that will be their reply - "tell Panasonic to fix its video footage".

So, the take home message here is there will continue to be a dance between the hardware vendors and the software NLE vendors where neither party will accept responsibility for this issue. And that is exactly how this problem has been playing out for the past 10-12 months.

Actually, as I said earlier, Sony does provide a transfer utility that correctly reassembles broken clips from their AVCHD consumer cams. I would imagine their footage would play back seamlessly in any NLE that supports native AVCHD footage. Unfortunately, I am not a Sony camcorder fan so that fact doesn't help me in any manner at this point.

[mcsmooth]

The file size restrictions imposed by the decision to use specific filesystems has nothing to do with the AVCHD committee. I contacted the consortium on this very point and did not receive confirmation that the AVCHD spec includes a requirement to (1) use filesystem "x" which would impose file size limits on video footage and (2) incorporate a software utility designed to sidestep this file size restriction and correctly reassemble camera clips that exceed any file size limit imposed by the particularly file system incorporated in a vendor's hardware product. Actually, they make it pretty clear that it is up to the particular vendor to decide how to logistically implement the AVCHD format, which, IMO, is one of the primary reasons we are left to deal with this problem.

Therefore, when you say it is the responsibility of the NLE vendors to follow suit (I assume you are implying that these NLE's are NOT AVCHD "spec-compliant"), your statement makes no sense because these NLE's are not "out of compliance." And that will be their reply - "tell Panasonic to fix its video footage".

The AVCHD spec may not REQUIRE a FAT32 filesystem, but did you ask if the spec is to recognize spanned files? Your logic does not make sense unless you can prove that the spec does not allow for stream files to be split physically and connected by the respective clip files. If that is the case (OR the camera does not do this properly), then you have a solid point. I just doubt it since some software was working before the camera came out and DVDs have been using a similar technique for many years which also use an mpeg2 stream and information files.

Your (2) would have no point if the files are indeed within spec. You wouldn't need a utility if compliant hardware/software is required to recognize spanned streams. People are asking for one because their software doesn't merge the streams properly, but the point is, it shouldn't be needed. I do agree it would be a useful tool though (even for people that don't have problems and simply want a single stream). There are a LOT of things I wish the AVCCAM viewer did, so I do hope for an update.

Keep in mind that the AVCHD "spec" is really just a modified blu-ray spec with some changed rules and restrictions. I do see your point about how there is some leeway on certain aspects that will give software developers pain anytime a manufacturer decides to do something different from what is already practiced. The major non-standards are in media, file structure and formats in the stream. The toughest aspect is the streams since software companies have been mainly testing with lower bandwidth 1080i files. File structure differences should be easy fixes whether they follow existing specs or not. What is being argued is if spanned stream and clip ability is assumed by all compliant players. I would think a good way to test this is to convert the file structure to blu-ray and load on a player that does not recognize avchd file structure. Those players are very strict to the standards, but will play back the streams when named properly.

The only way for Panasonic to "fix" their camera to the way people are requesting would force them to stop using FAT32. That would require a major firmware update if possible at all... I would expect that to never change on this model, it would piss off a lot of people. So unless they change or provide an optional second filesystem, files will never be under 4GB.

[ajcourtney]

No, I did not ask specifically about how the spec defines the proper method of handling spanned files. I don't think that thought process would have occured to me because all of us who were dealing with this issue were trying to eliminate the various clips by resplicing them back together to present a single file to the NLE. And no, I am not callilng for Panny to "fix" their cameras because I agree the hardware is functioning capably. Unfortunately, the incompatibility exists in a crucial area of software/hardware interfacing that enables each party to blame the other in such a way that nothing may ever get done to address this issue in a satisfactory manner for all users.

You mention that some software was working with spanned AVCHD files before the cam was released. That's news to me because everyone who tested this problem with Panasonic and Canon consumer cam AVCHD files experienced the same problem regardless of the NLE used. And converting out of AVCHD did not solve the problem either - I can personally speak for Cineform since I tried that format. Others tried whatever they had at their disposal - didn't matter.

I find it extremely interesting that Sony provides a utility to resplice these files back into the original contiguous clip to present to the NLE. Since they have taken this route and they run the show in Madison, Wis, I have serious doubts that Vegas will ever provide seamless editing of spanned files. And that's a real shame since I have been a Sonic Foundry user for almost 14 years and prefer that platform.

However, if this problem is resolved by converting out of AVCHD and into DVCPro HD, then I would have no problem doing that since I have always tried to convert out of AVCHD as soon as possible for editing purposes. Cineform provided that utility but was unable to resplice the clips, so the splice point problems were identical whether I was editing native AVCHD or Cineform. I doubt that DVCPro is as good of a codec as Cineform, but perhaps someone with some experience using both formats can chime in here. If it's good enough, then that's probably good enough for me.

[DaFireMedic]

Sorry to bring up the old thread, but I'm ready to buy (possibly) and need the info. I felt it better than starting a new thread.

I have $3500 in hand and as I mentioned am ready to buy the HMC-150, but I need to be able to shoot and edit seamless long form video. I read the entire thread but didn't really see a clear answer. I know that Barry said that the 150 is set up to record 12 hours of contiguous video, but I need to be able to get it into my NLE and edit it as such as well. I'm not concerned where the fault lies, only that there is an answer via hardware or software to get seamless transitions without losing frames between the 4gb clips and into my edited video.

Does the DVCPro conversion do this reliably so that I can have a reasonable degree of confidence in not losing any frames? Or is there another way?

If it helps I have Vegas Pro 8 and Premiere Pro 2.0. I am also willing to invest in CS4 if that would make the difference. Converting to Mac is not feasible for me right now.

Thanks in advance

[Averdahl]

Does the DVCPro conversion do this reliably so that I can have a reasonable degree of confidence in not losing any frames? Or is there another way?
Panasonics AVCHD to DVCPRO HD converter has passed all my tests i have done regarding this issue. PPro CS4 cannot yet "stitch" clips together properly. I hope an upgrade in the future takes care of it.

I do long shootings myself and i use the cameras HDMI to capture the footage with my Intensity Pro. Yes, the captures are realtime but i get seamless video so its worth it. :)

/Roger

[Barry_Green]

To reiterate a millionth time: the footage works. The camera works. The footage played back from the camera works. If there are issues, it's software issues and they'll need to be sorted out by the software manufacturers.

[DaFireMedic]

To reiterate a millionth time: the footage works. The camera works. The footage played back from the camera works. If there are issues, it's software issues and they'll need to be sorted out by the software manufacturers.

First of all Barry, thanks for replying.

I understand what you are saying and I read that before in your posts, but thats not what I was asking. I'm not complaining about the camera nor about Panasonic. Clearly the footage works in camera, which is fine if you are turning the camera over to your client and saying "here is your video", but most of us do not do that, we have to edit it and would like to keep our camera.

There is clearly an issue here (with the software manufacturers or otherwise). What I am asking is if there has been a software solution yet, some way of making sure that the clips can be made seamlessly contiguous in post, where it matters.

Averdahl: Thanks for the help, thats the kind of info I was looking for. This looks like a great camera and I am fine with converting to DVCPro (or using a BM Intensity card) as a workaround.

[floz23]

HOLY CR**!

I almost cr**ed by pants when I recorded my first shoot, spanning the 4gb fat32 limit. I put the two files into vegas and assumed they would span automatically, with no breaks... WRONG, either with 8.0c or 8.1. Thank the lord someone here suggested using the copy command to join the clips. The (windows) command line i used was


copy file0000.m2t /b + file0001.m2t /b + file00etc.m2t /b output.m2t

Whew!!! the resulting m2t, output.m2t, played back PERFECTLY in vegas. It seems that the 4gb limit will interrupt the GOP structure of the m2t, and the copy command simply joins the file, as if the gop were never interrupted in the first place.

anyway, i hope i save some other people on here from having a heart attack.

-Adam

[DaFireMedic]

Awesome Floz...Thanks!

Just to clairify (Its been a while since my last venture into the command prompt). I would just go to the command prompt, pull up the directory with the video files, then enter in the command line you posted (with the appropriate file names of course)?

Thanks again

EDIT: Yep, it worked just as you said. Thanks.

[Averdahl]

HOLY CR**!

I almost cr**ed by pants when I recorded my first shoot, spanning the 4gb fat32 limit. I put the two files into vegas and assumed they would span automatically, with no breaks... WRONG, either with 8.0c or 8.1. Thank the lord someone here suggested using the copy command to join the clips. The (windows) command line i used was


copy file0000.m2t /b + file0001.m2t /b + file00etc.m2t /b output.m2t

Whew!!! the resulting m2t, output.m2t, played back PERFECTLY in vegas. It seems that the 4gb limit will interrupt the GOP structure of the m2t, and the copy command simply joins the file, as if the gop were never interrupted in the first place.

anyway, i hope i save some other people on here from having a heart attack.
Adam, i almost got an heart attack when i discovered that it do indeed works! :)

This DOS command works as a charm and the output file works with no interruptions in Premiere Pro CS4. For PPro CS4 importing went smoother if i renamed the output file to .MTS.

Thanks a lot for your posting and lord someone for this great and valuable tip! :)

/Roger

[ullanta]

on the Mac, an equivalent command line (in the terminal) would be:



cat file0000.m2t file0001.m2t file00etc.m2t > output.m2t


I haven't tried this yet before an FCP Log'n'Transfer... but sounds like it should work.

[sagedrummer]

I tried using the above code with terminal but it didn't work. Have you ever got that to work on a MAC?

[ullanta]

What exactly do you mean by "didn't work"?

[ajcourtney]

it's a straight binary join command, so just use whatever the appropriate command is in mac os. the lame part of this workaround is you're going to have to evaluate which clips belong to a larger contiguous file and which ones do not. yes, the naming convention helps, but it still can be a tedious process nonetheless.

[TheGuitarzan]

As previously stated, Log and Transfer will not stitch the clips together properly.

TOAST WILL.

[sagedrummer]

What exactly do you mean by "didn't work"?
I tried the terminal code on the two clips and it didn't join them. However, my log and transfer in FCP has been working great and I haven't had any glitches so far.