Truly Corrupt Footage... ideas?

filmstox

filmstox

Well-known member
I've been working on a feature length documentary for a couple of months now and we've shot an enormous amount of 720@24pn footage on the HVX with great success. I'm importing the latest batch of footage taken by the producer while travelling. The producer has been handling a lot of the workflow and has been very careful in the way he handles the data. Somehow all of the clips from a single day of shooting are corrupt.

The Workflow looks like this:
3 4G P2 cards rotated to P2 Store
copied to Maxtor FW drive and dvd burn on a 12" PowerBook
imported to a Promax HD-SATA-e 2TB Raid on a Dual Duo G5
running OSX 10.4.4 and FCP 5.1)

On import, FCP 5.1 and FCP 5.0.4 reject the clip with the infamous "Clip incomplete or corrupt" error. Now the part that has me worried is that the clips really do seem to be a mess. The file structure and file sizes for the clips is normal but I get the following mess when viewing the individual files:

Corrupt Last Clip.txt:
0i;GÇ“?¿‘ÜÑ
Click to expand...

It should look like this:
Sample Valid Last Clip.txt
0037FK
1.0
38
Click to expand...

And the Thumnails all have the same garbage in the top 1/4 of the image...

00020W.BMP


The XML files are also a bunch of random symbols and text instead of the tagged text that make up normal XML files.

I haven't found out if the producer did anything differently when transferring footage on this given day, but I would love any thoughts on what might be the culprit and more specifically how we might recover the data.

Has anyone found a way to copy footage back to a P2 card? I'd love to see if the camera would be able to repair the clips or even manage to play them back.

Thanks,

filmstox
 
Last edited:
Do you have the ability to FTP one short (2s) clip?

We could try them and see what happens on a different set-up.
 
You should be able to copy the files back to the P2 card by just copying the Contents Folder back to the card. Make sure that your contents folder and last clip are all intact.

Good Luck,

Jan
 
Jan_Crittenden said:
You should be able to copy the files back to the P2 card by just copying the Contents Folder back to the card. Make sure that your contents folder and last clip are all intact.

Good Luck,

Jan
Click to expand...

Thanks Jan, I'm using the P2 store to act as a Card Adapter and the OS is convinced it is a read only device. The protect switch is off. I'll try loading the card in my XP laptop and try copying it there.

Have you seen this variety of data scrambling before?

Thanks again,
filmstox
 
Okay, I mounted the folder in P2 viewer on PC, could see the thumbnail but it wouldn't play.

So I tried an alternative method , I converted the clip using raylight (which bypasses P2 structure) to create an avi proxy that links to the original. Worked perfect, no corruption etc.

So the clip looks to be in tact but the access method to the file structure appears not.

Let me know if you want me to try anything else.
 
Last edited:
Excellent! Thanks, at least now I have hope that the day's shooting is not lost. I will have access to an Avid in the next few days and will try resurrecting the file that way since Avid doesn't use the XML files either.

Thanks for giving me hope!

Since the beginning of P2 I've always said, it's not a question of if, it's a question of when the system would fail. I see this as an inevitable fact, and for me the real question has always been how gracefully the system handles a failure. In this case it indeed seems that all is not lost.

I'm still going to troubleshoot this and try to figure out what went wrong and I will add some new steps in my quality control process.

My next step is to see if the camera can repair the clips...

I'll post more as I figure this out.

-filmstox
 
An update:

It turns out the DVD backups of these files were not corrupt. The error seems to have occured during one complete transfer between the P2Store and the Powerbook. There were 15 cards with the problem.

This reinforces my insistance on copying the DVD's from the p2store directly rather than doing the copy from the laptop. This would have only left us with a backup of the corrupt files.

Is there a way to enable a verification process in the copy from card/store to laptop? Perhaps P2Genie will answer this problem.

We still don't know exactly what went wrong but luckily we were able to put the robustness of our workflow to the test without loosing any footage in the process.

-filmstox
 
one nice thing about using a FW drive and haveing the camera control is it does checksum (I believe) verification on the files copied over - so you know what hit the deive matches the card contents.

_a
 
We've encountered the same type of video noise problem -even though we did a record check on the clips that were corrupt. The weird part is that the clip works for the most part, but does a color shift for about 72 frames of 720/24P then reverts back. At the point where I did the record check on the camera, I see the exact same thing Filmstox saw -except that it wasn't there when I reviewed the clip and moved on to the next setup. I only saw this once we took it into the NLE.

In addition, in other clips on the same 8GB P2 transfer the sound is digital static, not the audio we recorded. Again, it is arbitrary, but extremely frustrating.

I think people have to review each take before they move on -otherwise they will be in for a very expensive surprise -the re-shoot.
 
Joe Digital,

Something sounds wrong in your setup there. Don't know where you are but you should go to our webpage, click on support, look for a service center, put in the model number and zip code of where you live and the system will tell you whether to call the east or west coast service center. Talk your problem through with them and see what they say.

The behavior you are describing is not, I repeat not normal.

Best,

Jan
 
They will look into it. Unfortunately it seems as soon as the media leaves a P2 card I am told its very difficult to diagnose but they will try. They said perfect digital copies of whatever was recorded to the P2 is what the card is supposed to do so whether the problem is with the card or camera they don't know. They ask to try to recreate the problem, but in the meantime I will send them copies of the MXF files which were offloaded.

I was also told not to buy any more P2 cards and to consider the Firestore. I'm sure that's something you probably don't want Panasonic employees advising but they make compelling arguments.

This gentleman related there was a P2 viewer for Mac -is that true as I've not seen anything new as of this afternoon.
 
Joe_Digital said:
They will look into it. Unfortunately it seems as soon as the media leaves a P2 card I am told its very difficult to diagnose but they will try. They said perfect digital copies of whatever was recorded to the P2 is what the card is supposed to do so whether the problem is with the card or camera they don't know. They ask to try to recreate the problem, but in the meantime I will send them copies of the MXF files which were offloaded.
Click to expand...

In your work flow, you should always transfer the MXF files to an HDD before you try and import to FCP. This is a recommended workflow from Apple.

I was also told not to buy any more P2 cards and to consider the Firestore. I'm sure that's something you probably don't want Panasonic employees advising but they make compelling arguments.
Click to expand...

I would like a Private Message with the name of this individual. Frankly while I feel the Firestore is a nice product, it is not the perfect product and I fully believe that the P2 Card is vastly more reliable as it has no moving parts. It may be that he heard what you were trying to do, and then made a recommendation, but I would always recommend P2 first and foremost, then Firestore is the application was one that warranted that decision.


This gentleman related there was a P2 viewer for Mac -is that true as I've not seen anything new as of this afternoon.
Click to expand...

Imagine Products, www.imagineproducts.com offers a P2 viewer as part of the HD Log software. Very cool and lots of flexibility.

All the best,

Jan
 
Well, those darned, darned P2 cards. But there's a valuable lessen to learn here: Next time you are doing a job for paying customers, don't cut corners at the wrong place, instead get some _reliable_ storage solution like the Citidisk HD!

;)
 
You must log in or register to reply here.
Back
Top