Disappearing MXF files -Help!

R

Replicant

Member
Been working with the HVX for almost a year. ZERO problems thus far. Last week I had a client rent the camera for 15 shooting days straight on location. Needless to say, i had to come up with a good way to backup and archive the footage on location.

Ended up using P2 Genie to backup the P2 Store to TWO firewire external HDs hooked up to a powerbook. Redundancy. this was done every night -- followed by a clip-by clip viewing of the backed up footage. THEN the P2 store was released and formatted.

NOW the problem. Out of the 115 P2 Volumes/Cards consumed by that particular project, there is one card that has missing MXF files! P2 Log Pro shows 22 Clips complete with thumbnails and metadata for that card/volume, but clips 8-22 simply wont load. and if you go into the CONTENTS>VIDEO folder, there are only 7 mxf files. But the CLIP, AUDIO, and ICON folders all have 22 references! So the MXF files disappeared.

I say disappeared bacause the files were played back and verified using P2 Log ON LOCATION, and they were all intact. Whats more, the 2nd BACKUP HARDDRIVE also had the same missing files... so i can't really attribute it to physical damage to the Main Harddrive.

Anyone have similar experiences? Any information or GLIMMER OF HOPE is greatly appreciated! :happy: :happy: :happy:
 
Anyone?:embarasse If i cant find the missing files, we're gonna have to re-shoot some sequences. Theories welcome as well....

Thanks!
 
Just out of curiosity, when you did the clip by clip viewing after copying the files every night, did you still have the P2 store connected to the computer? By your wording in the above post this could have been the case. If so this could explain why you were able to view the clips even if you were importing P2 from the HDD. If there are reference files for the clips in folders on the HDD but no actual media, they could have been sourcing the media from the P2 store if it was still connected.
 
Thanks for the replies...

Using a Powerbook G4.

The footage was checked every night for two reasons: verification of transferred footage and the director and staff also wanted to view the rushes for the day. to help them plan out the next days. worked out well.

the p2 store was definitely unplugged during viewings, this has been one of the ten commandments for us from the time we started using P2 Log to check footage.

Anyone have this experience?

I know the technology aint perfect... what boggles me is how PERFECTLY IMPERFECT this glitch was... vanishing identical files on separate HDs -- leaving all metadata, icons, audio, intact. almost as if someone had gone in to the folders and deleted the mxfs! hahaha TWICE !! Saboteur!

guru Barry? anyone?
 
I have never, in thousands of cards, had any case of any disappearing files. Ever.

Were these 8GB cards? If so, it's possible that the icons and such were stored with one partition, and as many video files as would round out that partition; then the remaining video files were stored in a second partition.
 
The only other trouble shoot I can suggest is to see if the files ended up on the internal drive of the G4 for some reason.
Other than that, I personally have never had this problem although I have heard of people that have had media files in FCP dissappear overnight. ie. they were working on a sequence, saved and closed down, then get up the next morning and media is offline and missing from the drive. This seemed to happen when the drives were approaching full capacity.
 
bet it does have something to do with almost full hard drives- I have heard of vaguely similar problem where p2 genie acts like it is copying footage to a drive, but drive because it is already too full has no room and while things look like they are going over- stuff doesn't. may be a mac os thing rather than p2 genie thing. I have had at least once where i tried to pull stuff off of p2 store and one partition (somewhere in middle if memory serves) wouldn't copy over properly. copy time was abnormally long and when i went to verify with p2 log I had metadata and icons but no actual video. boy was I glad i tried to play each clip, otherwise I would have been very unhappy later. Tried reimporting, continued to have problems with that one partition, but eventually got it off and all was well. BTW i tried doing disk image of that one partition- and it would give me a faiiure warning, so while it didn't sucessfully pull it off, it confirmed there was an issue. Anyway, i can't remember how i solved it but it finally took.

H have run diagnostics on both my p2 stores every 25-30 hours of use (on my PC with panny software) never had an issue, and also reformatted them same time. Not had a problem since, or any indication of trouble with diagnostics.
I totally trust the p2 cards, I pretty much totally trust the p2 store, insofar as card to store, but from p2 store to computer, or p2 card to computer I totally don't trust things. So much for mac being multimedia friendly. frankly i expect problems- from cards freezing the machine to mysterious mxf disapearences.
Frankly I am thinking of shooting a youtube video take off of mac commercials only pc guy in business suit has a neat folder of files, and turns to mac guy and says, "How is the file copying going?" and mac guy as stoner confusion dropping papers and rumpled notes "dude, i think i copied it, I dunno.... it was three files, right? or was it four...?" PC guy" it was five files. you have it all right? it is important stuff you know." Mac guy; "Five? Dude, I don't know, maybe.... how important can it be?" PC guy rolls eyes, fade out.
Got to fix these gremlins soon please! I like the camera, and like the concept of the workflow but just too many gotchas! and none have been satisfactorily resolved.

to recap problems (with mac)
1) p2 cards sometimes freeze mac
2) sometimes p2 store does not mount all partitions and does not tell you of failure
3) 3rd party software had to be created for multiple partitions on p2 store for offload- but seems to be buggy- but all fingers point to mac os errors- mainly copying that doesn't actually copy either due to full hard drive or "making" folder in p2 genie rather than making it first on drive using mac os. either nothing gets copied or wierd things like no mxf files happen....

am i forgetting anthing?
 
I will add to the list. One of the 3 cards I shot this weekend never made it through the laptop to Mac process. I can view the contents and lastclip folders but they will not load into the P2 import function of FCP.

Also, on the two cards that did load, several of the clips cannot be added to the que. They all appear in the import window and can be previewed- but never appear to be used in final cut. The problem has something to do with renaming - if I just click on the icon and leave the number there instead of renaming, it usually imports (but not always).Of the ones that refused to import - it doesn't matter how many times I try to re import, it will not go. Very frustrating because I am looking at previewable, playable files - but can't use them in any way.

This has only been an issue when using the laptop to store clips and then transferring them to the Mac. I have uploaded cards straight from the camera many times, via firewire, and never once had an issue.
 
Great input, folks. As of last night, the director and I have accepted the fate of our missing shots. We've conceded and attributed it to spiritual intervention, which will prove to serve a purpose in the future.

For the sake of properly documenting this case for the benefit of anyone else unfortunate enough to run into this situation in the futre....

-We used 2 x 4gb cards. and we never shoot with both cards inserted. i don't like the idea of spanning clips.. and it allows us to rotate the cards effectively during shooting.
- P2 Genie has that behaviour of DUMPING onto PHANTOM folders/drives. It happens when you designate a destination that is either 1) non-existent or 2) lacking in space. for case number one, it usually happens when you forget to re-designate your destination from a previous session -- usually an external device -- and Genie tries to use that same folder. when you hit copy, Genie WONT TELL YOU your drive no longer exists... it will instead create a phantom Volume on your computer/laptop. these phantom volumes are usually in "computer/.volumes/". they will neither mount nor show up in finder together with your real volumes because the ".Volumes" folder is invisible. you have to be able to see invisible files before you can dig up these phantom volumes. these volumes will have the same name as you original destination.

I checked the phantom drives on my laptop. I only found data from actual bad dumps, coming from way back when I first installed P2 Genie and stress-tested it.

- on bad dumps... For those using my same process (P2 Store to FW Backup drive using P2 Genie)... be careful with your destination device. If you use an external enclosure that has faults with, say, power... it'll freeze your transfer process, resulting in bad dumps. Some external devices "choke or disconnect during data access... not enough for OS X to detect, but enough to botch up your transfer. I know this because i live in Manila, where external enclosures from china and taiwan cost as low as $10. they're cheap for a reason.. the slightest fluctuation in electricity disconnects them. i don;t use them.. i go with the $12.95 enclosures!!!! :)

Why use $10-dollar enclosures with a 10K camera kit? hahaha COST! for the price of a big-ass Lacie, i can have a whole bunch of 320GB seagates in Transcend (taiwan) enclosures. Some call it crazy, i call it Risk Management. !

seriously, the enclosures i used for the above-mentioned project were $100-types. it wasn't a drive fault, for if it were, I wouldn't have 112 cards worth of GOOD footage. and if the enclosures were at fault, it still wouldnt explain the identical damage to two very separate drives.


Maybe i should start a thread called "how we do it the philippines... Third World HD"

Thanks to all!
 
Last edited:
Thanks for the detailed post regarding the disappearing MXF files.

It's a good warning and something for us to be aware of.
 
You must log in or register to reply here.
Back
Top