Firestore Nightmare...What else can I try?

M

mtd2007

Active member
I have been troubleshooting some footage from a two-camera shoot where I used the FS-100 for the first time (one on each camera). It was sort of a necessity because we were shooting a 2-hour panel discussion with a pair of HVX200s and there were no breaks to be had for switching out P2 cards (and not enough money in the budget to rent 32GB cards).

Did my homework (or so I thought) up front, enough to know that the Firestore can be tricky and requires proper settings/etc., especially when spanned clips were to be expected.

As such, I tested my planned workflow until everything went smoothly. Of course I exhibited the exact same smoothness on the day of the shoot but now very little is working properly in post.

Cameras were set to 720P/24P as instructed in numerous manuals and guides...Firestore was set to record 720/24PN in P2 format. I organized the files before export and when that alone wasn't enough I repaired them as well (this is what worked for spanned clips when I tested).

I unfortunately was working with rented Firestores and ran out of time and money while troubleshooting...so everything that I'm about to list below comes with the caveat that my source footage has now been dumped to an external harddrive (also backed up).

Working in FCP 5.14, MacBook Pro 2 gig Intel Duo, and the plan was to import the footage from my 250 gig USB 2.0 Western Digital drive to my Firewire RAID.

In the past, I have used P2CMS and then FCP for import and have run into zero problems. This time...

1. I was receiving red Xs in P2CMS and couldn't play any of my spanned clips - fixed this by removing the "Invalid!" data from between the VFR ratio tags in the XML files, and this seemed to fix things until...

2. I tried to import into FCP. At first pretty much nothing worked right, but eventually I was able to import almost half of my clips safely by applying some of the solutions listed here: http://dvxuser.com/V6/archive/index.php/t-78488.html

3. The remaining problematic clips would not import through FCP's Import Panasonic P2 window (stop sign: error unknown) and while I did manage to import the MXF files directly into my logging bin, when I went to match them up to their audio files on the timeline....the audio files abruptly ended mid-way through the video clip...so either my remaining audio is nowhere to be found or somewhere I can't get to right now.

4. Tried a Raylight demo, it worked for my "good clips" but my bad clips stayed bad - Raylight would convert them but the video would go to hell (every other frame practically destroyed, or sudden cuts to solid green or black, depending on the clip). I fiddled with different settings with Raylight but nothing seemed to work.

5. Couldn't really accomplish much testing with P2Log or HDLog because the demos only import the first clip...and my first clips are neither damaged or spanned on either volume I copied from the Firestores. Attempted to place a damaged clip "at the front of the line" but didn't know quite how to do this in the end without messing with the files too much.

6. Tried copying the CONTENTS and Lastclip.txt files over to a newly created folder on my RAID...still no dice. Same problems pretty much.

Actually...I've tried just about everything I could think of...or could find by scouring the web.

A few things I noticed, not sure if they are significant.

1. The clips that work (after some troubleshooting) have extra data in their XML files...maybe this code has something to do with "properly" spanned clips? Not sure if the clip names referenced in the code below show up in my files, though...

<Relation>
<OffsetInShot>57344</OffsetInShot>
<GlobalShotID>060A2B340101010502010D431300000000031516022D2F35E416EA0200002332</GlobalShotID>
<Connection>
<Top>
<ClipName>0040FJ</ClipName>
<GlobalClipID>060A2B340101010501010D431300000000031516022D2F35E416EA0200002332</GlobalClipID>
<P2SerialNo.></P2SerialNo.>
</Top>
<Previous>
<ClipName>0046AJ</ClipName>
<GlobalClipID>060A2B340101010501010D431300000000033739022D2F351D6BEA0200002332</GlobalClipID>
<P2SerialNo.></P2SerialNo.>
</Previous>
<Next>
<ClipName>0048HJ</ClipName>
<GlobalClipID>060A2B340101010501010D431300000000040714022D2F35F822EA0200002332</GlobalClipID>
<P2SerialNo.></P2SerialNo.>
</Next>
</Connection>
</Relation>
​
2. The "missing" audio in the bad clips seems to go missing at the same interval throughout. It appears to be otherwise synced to the video (until it disappears). This audio is not just missing when I import the MXF files directly into FCP...also drops out within FCP's P2 Import window and in P2CMS.

I'm a long-time visitor to the site but a first time poster...and I know the above is obscenely long but...can anybody help me? Is there anything else I can try?

My plan for now is to get my footage to a machine with FCP 6 to see if things go better...or to buy a third-party app if anyone recommends it...but I'll be completely honest and say that money is unfortunately a big stumbling block here. I'm saving up for FCP Studio 2 but I can't rush the upgrade either. I'm in the middle of editing an indie featurette of my own and I don't want to jeopardize my setup.

I don't understand why, if all of the clips were created by a single start-stop sequence on the camera...only half of my clips seem salvageable!

Many thanks for any advice!
 
Hi Noel,

I forgot to mention that when listing the many things that I tried - yes, that was one of the first things that I attempted, both within FCP and when testing Raylight.

Didn't seem to affect anything either way, unfortunately...
 
Am I a lost cause? Anyone know if maybe FCP 6 will magically make my problems go away?

Or is it worth it to buy something like HDLog to try to solve the problem?

Any advice GREATLY appreciated.
 
Thanks to anyone who at least read this thread and gave it a thought or two. The prognosis seems to be that the missing audio is gone.

Focus Enhancements finally returned my messages about a week ago (after I started getting testy) and their attempts at customer support were insultingly un-helpful.

Just wanted to write in quickly for future thread-viewers to let them know that 1) The Firestores are not the best pieces of equipment out there (many videographers and editors that I have turned to for help groaned as soon as I even mentioned that I used them) and 2) They are a particularly bad choice for an event shoot such as this one...where spanned clips...or one massively spanned clip are to be expected. The best guess that I (with much help) have been able to come up with in terms of what happened is that there is a difference between asking the drives to handle a small spanned clip (made up of two or three smaller segments) and a giant spanned clip (made up of many segments).

Maybe everyone already knows this and I'm just a poor chump. Ah well.

An expensive and embarassing live-and-learn for me...should have worked something else out for this shoot.
 
Last edited:
I don't really get what happened... so you are saying both drives did the same thing.. so there's no audio/video from either camera in sections.... ???

So you shot for 2 hours without a single button off/on... how big are the files???

2 hours at 720pn should come it at around.. 60gb a pop.... are your files that big???
 
Hi Lez,

Yes...both drives did the same thing. All of the error clips that I was unable to get into FCP through any other means but a direct import of the .mxf files are missing audio at the same exact point on each clip, from each drive.

Basically, after all my troubleshooting, for any given 10 min stretch, I have a 5 minute spanned clip that works completely, and then another five minute chunk (from the error clip) that drops the audio at about two minutes in (but not the video). This pattern repeats through the entire set of footage, for the most part, with few exceptions.

And the total number of gigs for the files is coming in under fifty for each drive...because the .mxf audio files for the error clips that should be 30 megs are closer to 10 megs. It seems to me that they probably were corrupted during the spanning, or when I ran the P2 organize or repair functions in the Firestore to try to get them out of the drive after they weren't importing properly.

Guess they could have just not been recorded in the first place either, but that would mean that audio was dropping, picking up again, and dropping as time went on. Maybe that's possible...I don't know.
 
All sounds weird... have not used a firestore before.. but I had an issue last year when I shot an 4 hour event....

FCP would give me errors on import.. in fact it would not import... I used P2 Log Pro to convert the MXF's to Quicktime and all worked ok in the end... although I had an annoying 1 frame overlap with each clip..

But it did end up bring the clips in for me...
 
It definitely is weird. I've spoken to many other professionals by now and that's exactly the sort of reaction I'm getting.

I would love to have a one-frame overlap at this point. And I would try P2 Log but just am not sure it would be worth the money since it just appears that the audio is straight-up missing.

Everything points to something going wrong within the Firestore workflow. I'm assuming your event was shot on P2 cards?
 
Unfortunately it didn't. A local post-house was kind enough to let me drop by with my footage to test it on their FCP 6 station, which I believe is up to date.

Their head of post-production actually stopped for a few minutes to try to help me and between us we still couldn't figure anything out. The audio files themselves just seem to be cut-off, however it happened. And I literally went in to to the CONTENTS folder and accounted for all the clips...just to make sure none were floating around and getting lost when FCP or P2CMS went looking through the .xml files to span the clips together.

That's when I discovered that the audio files for the problem clips seemed too small in size by two-thirds.

I'm currently pushing forward with an edit to see what can be salvaged from what I do have...but thanks to everyone for continuing to suggest fixes. I'm obviously listening because it's still killing me to have lost the work.
 
1. I was receiving red Xs in P2CMS and couldn't play any of my spanned clips
Click to expand...

This is were I get confused. As I never get "spanned" clips from my firestore.

Just for the record, and this is only to be 100% sure. Spanned clips occur when when one p2 card becomes full and the camera switches to the next. So when you import you get the clip with (spanned). Thus a standalone firestore will never get a spanned (clip). Unless you had two firestores running on one camera.

So now thats out of the way. Can you please give some more info on your "spanned" clip.

Also do you know the firmware version of the fs you used?

I have been using my FS non stop for quite some time. Never had an issue with anything related to it, and I think the workflow is fantastic. Thats not to negate your issue. Im just wondering if theres still a solution.
 
Hi Noel,

A lot of this I'm just relaying from research, and excuse me if it's stuff you already know, but here's all I can figure.

I guess because of the 2GB file size limitations of the FAT32 format of the Firestore drive, the device takes successive clips and "spans them" once they hit the 2GB mark. So what I should have is a succession of files around that size that follow each other sequentially. Even if they don't show up as spanned in FCP.

And that is almost what I do have, now, for my video, after dropping the .mxf files from the problem clips straight into my timeline in the right order.

The firestore was rented, but I did check the firmware before I used it and saw that it was running version 4...unfortunately I'm not sure if there were any other numbers added to that. I checked this to make sure the devices would allow me to shoot in pN mode.

I should mention, though, that when I perfomed my test for this very issue (minus, of course, the missing audio, which was not AT ALL expected) the test clips did show up as spanned after I ran an Organize P2 function, and then repaired the clips within the Firestore.

That exact workflow:

1. Shot for several minutes in 720p/24pN until I was sure to have successive clips.
2. Ran P2 organize. Then ran Repair Clip for each of my clips.
3. Connected FS to computer per the manual's instructions.
4. In FCP 5, opened the Import P2 window.
5. Clip showed up as one spanned clip that imported and played with no problem.

As an aside, I should mention that I performed this same exact text shooting in Quicktime PN. Although it worked, six full seconds of video was not recorded between the spanned clips. I know this because I was perfoming my test by shooting the world clock on my computer screen! No offense to anyone who has had success with it, but that might be another FS warning.

So you've recorded and ingested long, large clips using the Firestore with no problems? Maybe something with my workflow was just plain wrong, despite all the testing. Either way it's a little too complicated for me considering how much easier using P2 cards have been.
 
I've put a lot of footage on my firestores and never once experienced the problems you describe. The worst thing that's happened was a disconnected cable which I could have avoided by taking a little extra precaution. Fortunately that was not during a critical moment.

I've captured many 20+ minute clips, a day's worth of time lapse and numerous other takes running all day long using two cameras/FS-100's. They have never let me down.

I hate to say it but it sounds like you had / have a configuration error, bad cable or other gremlin that prevented the data from being recorded properly.
 
Yeah...that's what it sounds like. Just figured I'd record everything here in case someone else is looking at a similar situation and wants to take precautions.

Of course I'm glad the drives work for some people, but I have obviously had a bad experience and to be honest the customer support from Focus was really poor.

Again...thanks to everyone who tried.
 
Ok first off, sorry to mislead you. You do indeed get spanned clips on FS (as per your description). I just went through my projects and wallah there they are. Just never had an issue with it so never really noticed (on todays list of things I never noticed before).

Im wondering why you used "Repair clip" in your workflow? Wonder if that had some effect.
 
The Repair function is on my worry list of things that may have gone wrong. It always makes me nervous to perform any sort of process on a file like that. Maybe my first mistake was not backing up the footage before pressing a single button. Although it really shouldn't be that easy to screw up your footage if that's what happened.

I did the repair after noticing a list of troubleshooting steps for spanned clips in FCP that was included with my manual when I rented the drives. Basically it suggested that running the repair would fix the issue - which it did during my tests.

Then again, just to be sure that this wasn't the problem...I held off on repairing the clips on one drive while I worked off the other.

Who knows....
 
Yes, Im stumped Im sorry to say. But, just to confirm, in log and transfer on the real footage (not test), did the clips work as normal?
 
You must log in or register to reply here.
Back
Top