File Corruption Advice Needed

[rwatkins]

So, a producer's worst nightmare is losing footage. I love the HVX200 and our company has three of them. But today, I miss my tape...

Our workflow is as follows:
8G P2 cards transferred to P2 Store on location.
When we get home, we copy the cards to two sets of external backup drives via drag and drop.

Today I started using HDlog to prep for my editor. When I got to the fifth out of eight cards, the playback was your standard multicolored pixle noise and static for audio for three out of the four clips on the card.

No thumbnail pic was visible in the HDlog clip window for any of the clips, but timecode and in's and out's, file names, etc. were all there and the first clip of four total plays fine.

We have at least five days of shooting and BROLL days in other folders on the drives that play back fine and pretty. It seems to be only these particular files from this particular day.

We rotate camera packages and I know we used this particular package twice. I know that because late in the video noise on one of the corrput clips, I see a previous day's shooting blinking in and out. We erase cards in the P2 as we cycle them through the camera and the P2 drives are wiped after the footage is transferred. Its also my understanding that you cant record over clips on a card.

So before I sacrifice another weekend to the video beast, I was wondering if anyone has encountered a similar problem or discovered a solution. I'll be trying all the advice that's already posted here as well as applying my own intellect. Give me your advice and Ill keep my findings posted in hopes of helping others in this situation.

Peace

[rwatkins]

So this morning I try the following fix for the corrupt files...

First a quick prayer to the gods, then transfer the files from the La Cie Mac formatted backup drive to a La Cie Fat32 formatted drive so I can use the P2 slot on my Vaio to transfer the files back to the P2 card in an attempt to repair them on the camera.

As I was doing this I noticed a couple of things. First, the Video folder had more clips listed in it than the three or four I saw listed when I imported the footage into HDLog. I also noticed that the last clip.txt file was composed of symbols and not legible text as had been noticed by a previous poster with a corruption problem.

With trembling hands I inserted the P2 card into the camera and set the camera to P2 playback mode. When I looked at the thumbnail screen, only blue windows showed up and the card read empty (8Min on an 8G card) even though I know the files are there. I went to the repair function in the submenu hoping this would fix the problem, but the REPAIR CLIP option was grayed out. CRAP!!

So, I've emailed Panasonic Tech Support and am trying to remain hopeful. If anyone has successfuly repaired corrput footage on a card, I would love to know how you did it.

Peace

[Justin Marx]

So this morning I try the following fix for the corrupt files...

First a quick prayer to the gods, then transfer the files from the La Cie Mac formatted backup drive to a La Cie Fat32 formatted drive so I can use the P2 slot on my Vaio to transfer the files back to the P2 card in an attempt to repair them on the camera.

As I was doing this I noticed a couple of things. First, the Video folder had more clips listed in it than the three or four I saw listed when I imported the footage into HDLog. I also noticed that the last clip.txt file was composed of symbols and not legible text as had been noticed by a previous poster with a corruption problem.

With trembling hands I inserted the P2 card into the camera and set the camera to P2 playback mode. When I looked at the thumbnail screen, only blue windows showed up and the card read empty (8Min on an 8G card) even though I know the files are there. I went to the repair function in the submenu hoping this would fix the problem, but the REPAIR CLIP option was grayed out. CRAP!!

So, I've emailed Panasonic Tech Support and am trying to remain hopeful. If anyone has successfuly repaired corrput footage on a card, I would love to know how you did it.

Peace

Did you rename the lastclip.txt to the proper name (without the symbols?)

[rwatkins]

Hey Justin,

Thanks for the advice, thats the next trick I'm about to try. I found another thread that seems to tell me how to do it. If it works, you can be sure I'll post complete instructions for everyone in the community.

thanks

[rgbuser]

Just a thought but you've probably covered it: when you tried to repair clips in camera, was the format the same as for the shoot? As you probably know, you need to match shooting format with recorded format to do things with clips in MCR mode. And the best of luck. Your story is sending shivers down my back.
Tom

[rwatkins]

DAY FOUR

OUTSTANDING advice Tom. I had initially tried to repair the clips in the camera and they came up outlined in red. As the manual states, clips outlined in yellow can be repaired. Those in red cannot. However, out of the 16 clips on the card, three will read up in HD Log and out of those three, one is usable. These three are outlined in yellow in MCR mode. The other two play, but only multicolored snow. I'm going to go and make sure the settings are the same in the camera as what I know the clips on the card to be and hope for the best. More info soon.

Oh, I did remake the LASTCLIP.TXT file and save it back to the card, but I think the poster was right in saying that this file is only critical if your loop recording over two cards. I knew it couldn't be that easy. No contact back from Panasonic Tech Support yet, but its only been one day. Ill give them a few more before I give up. More to come in our exciting saga... stay tuned.

[rwatkins]

DAY FOUR CONTINUED

Just as I thought, we only shoot 720P/24PN. All our cameras are setup that way and dont change. I tried to repair the three clips that seemed like they should be able to be repaired, but the camera wasn't into it. Ive examined the contents of the files and here's what I'm thinking:

The flie sizes are correct and consistent, so my data is there. I'm willing to bet that its the XMF portion of things that is messed up. My understanding is that the XMF carries the data in relation to clip settings, frame rates, time code, etc. Ive noticed that the three clips that will import into HDLog and P2 Viewer have time code and other information present in the fields as it should be. These files also show 720/24pn DVCPRO50 and Duration in the thumbnail view in the camera, the other 10 do not. My feeling is that if I could somehow access the XMF data, I could re-input the settings and fake time code. Basically fake the XMF data and be able to open my clips. I don't relish the thought of spending the money for a second reshoot day. Is there a program that will let me view the XMF data?

I've also contacted another local production company who uses avid instead of FCP. Its my understanding that avid could care less about XMF data and might open the clips on my card. I always like avid better anyway.

So, as of this point, nothing is resolved and my bar tab is waaaaay up. Still looking for any suggestions from people who have worked through or around this issue.

[rwatkins]

SOMETHING INTERESTING

So in desperation I though "What if I delete the old XML files and replace them with good ones from other clips. Would that give HDLog or FCP the info it needs to play my clips?" So, on the Mac I unlocked and then deleted the XML files in the CLIPS folder. I took an XML from a clip that plays fine from a previous days shooting, unlocked it and copied it 13 times, once for each clip in the corrupt card folder. I then renamed the copied XMLs with the clip names from the video and audio folder and placed them back into the CLIPS folder. I then imported into HDLog. This time, instead of 3 out of 13 clips coming in with only one usable, All 13 clips came in, but none of the 12 that have problems played anything other than multicolored snow. Of course, it thought all the clips were the same name and had the same TC info etc.

I know this doesn't solve my problem, but I thought it was interesting. Is there a program that will let you view XML data? Would MS Access do it? Where is the corrupt information? Is it in the actual video and audio data or is it in a file that interprets and translates it for whatever system you are editing on? This isnt fun. Be VERY CAREFUL of your data brothers and sisters. I would not want anyone else to go through this experience.

[Shazam]

Any web browser will read XML data. Just open the file in IE or Firefox.

[Shazam]

I just re-read your intial posts. Don't know if this will help much but I think the corruption issue may have occurred becuase your back-up drive is mac formatted and the other drive is FAT32. Seems like I recall macs smetimes have an issue when converting between formats. Do you still have the original files that you transfered to the P2 store? I think these are what you want to be working with since they were copied as FAT32 to FAT32.

Fingers crossed for you.

[rwatkins]

thanks Shazam. Ill try opening them later today in a web browser and see what they look like. The file corruption happened either between the P2 card and the P2 store or the P2 store and the backup hard drives. The FAT32 drive was only used to transfer the files so I could work in PC environment to put the files back on a P2 card. The files are really backed up on a set of Mac-formatted La Cie 160GB hard drives. Nice little drives BTW.

[Shazam]

Cool. I just ordered their bus powered "Little Big Disk" to use which sounds pretty awesome. (Haven't gotten it yet.) External RAID0 and no power adapter required!

BTW, just to be clear, I highly suspect the point of failure to be from the P2 store to the LaCie back-up. The P2 Store is Fat32. When you made the copy I am guessing OSX corrupted it somehow during the formatting conversion. I have heard of this happening before.

Also, as a suggestion. Have you tried using FCP to import the files off the back-up drive? (I'm assuming they're already deleted off the P2store. If not, do it off that instead for sure.) FCP 5.1.1 can import right off the P2 card (or copied file location). May be a long shot, but perhaps it can figure out what the P2 log can't.

Here's a link to how to do it in case you need it.

http://www.creativecow.net/articles/ross_shane/p2_workflow/index.html

[rwatkins]

Hey Shazam,

Yeah, we are pretty sure thats where the corruption took place and of course the P2 store was wiped clean. Lesson learned on that one. I did try to import the footage into FCP, but it crashes the system so no luck there. I'm still going to try to open the XML files and see what's in there. I'm also calling around San Diego to see if anyone has an AvidHD system I can try to import in on. I may have to drive up to LA for the day as everyone here seems to be Final Cut or Avid non-HD. As usual, Ill keep the community posted on the outcome. At least my co-workers have taken all the sharp, bladed objects off my desk.

[Shazam]

Damn. Again, good luck.

[rwatkins]

THE FINAL CHAPTER

So, after two weeks of trying everything I could to fix my footage, nothing worked. Even taking it to other systems did not work as they could not recognize or import it. Basically it comes down to that during transfer from the P2 store to external backup drives, the footage became corrputed due to a jiggling of the cable, power surge, unclean power or just sheer capriciousness of the gods.

So now I have to spend THOUSANDS of dollars to reshoot a handful of narrator on camera paragraphs. What a pain in my @#&!!

I must say however, that throughout this process Panasonic's technical support was BEYOND PATHETIC. It took me three emails for them to get back to me and when they did it was with the following message "Insert the card into the camera and use the repair function to fix the clip." Its obvious that Pana doesn't even want to acknowledge the problem or deal with it. I've never heard of anyone successfully repairing clips in their camera and am starting to believe that its all BS anyways. What, exactly, would the camera be able to repair? And if it can "repair" clips, why couldn't it repair mine? Once again, Pana tech support failed to even return my emails on the subject and with no phone number to call, its clear that they dont even want to hear about your problems... Its the phone company... No, its your internet provider... No, its your cable company... Where have we seen this before??

So, all my producer/director/shooter friends... if your footage becomes corrupted, you are on your own and can go cry in your drink because you ARE SCREWED and you dont even get a kiss first. Good Luck to all of you. And once again, shame Shame SHAME on you Panasonic for not offering any kind of tech support what so ever. Its truly unacceptable. Im off to get pissed. Peace.

[Anders Holck]

Last option would be to try the Raylight software, but it's only for windows. I have had some luck convertion some corrupted files in the past that wouldn't work in either FCP or P2 log.

[GroovyBrent]

Hey All -

Robert (rwatkins) is my Senior Producer, and we're in the process of scheduling a reshoot to replace the corrupt footage.

But just for giggles, I thought I'd try one more thing...

At our website (http://www.groovylikeamovie.com/corruptfootage.zip) is a .zip file with the corrupt footage in it. If anyone wants to take a look and see if they can repair it, I certainly would be grareful! One clip plays fine, the other 2 are bad.

As a bonus, if you happen to figure out how to repair the footage, I'll tell Robert to give you a big sloppy kiss.

[rwatkins]

As long as you're a cute girl or woman, preferrably 30-40. We are booked to re-shoot on Oct 11th, but if anyone figures it out after that, we would still love to know.

[bpfox77]

We had a shoot over the weekend. the first clip on the card starts off good for about five seconds and then multicolored snow and white noise. i had formatted the disc, but after the initial five seconds, before the noise, there is a frame of the previous day's shoot. (which obviously should not be there since it was formatted). All 20 minutes are gone.
doesn't sound like a lot, but it is, I had rented a police car for the scene, 4 actors, etc.
I've tried all the suggestions made here and p2 log pro can not help.
the card itself is corrupt, i'm certain it has nothing to do with transferring to drives. I still have it, the camera cannot read it? It wasn't pluggged into anything, don't think it was a power surge. I'm thinking it has something to do with the write protect switch.
I did a test with one on -and if you turn the camera on and quickly begin recording, it goes for about five seconds before telling you the card is protected.
what fun.
it was a small crew, (no script notes) so it's hard to put together what we lost of of six cards of footage. I've got decent coverage so hopefully no reshooting.
I have been using and loving the p2 system a ton til now.
based upon other threads i've read, seems like panasonic will not be able to help. so . . .

[Justyn]

I have corrupted files that play in the p2 import in FCP but won't get converted to quicktime files... Say they are missing data and don't do anything. Anyone have some advice over this. the clips look fine and all the data looks the same but they don't import. Would P2log or hd log be better choice? I'm willing to try anything.


Thanks

justyn

[Justyn]

P2 log didn't seem to work... So frustrating. The files play fine in the import and they are there.. but I can't convert them to quicktimes.. What could be the problem? I backed them up to a HD as full cards.. I won't delete and hope I can fix them either in the comp or the camera. I'll have to contact Panny... So FRUSTRATING>>> now