Corrupted files from Sony A7S iii

[MrP]

Hi there,

Long story short, I have a handful of .mp4 files from a Sony A7Siii shoot that are unplayable anywhere. MediaInfo doesn't even show information for them. A few before and after clips work, but the lot of them are all unplayable. I found RepairIt by Wondershare but it has so far been unable to repair any of the files. Are there any other solutions out there that can "fix" .mp4 files? They all have data (some are 3+GB) but none are playable. The files are on the PC. I don't need to recover deleted clips but rather somehow "fix" these...

More information:
I have a card that had over 100 clips shot on it in various formats (10bit, 8bit, 23.976, 60, etc.). I always shoot dual card recordings in case one card fails but in this case I saw no failure, copied the card to the server using TeraCopy (using PC), TeraCopy did the verification (not sure what method is used), no errors. I always save the cards until needed again at which point I format and reuse. Of course this project was on the back burner so I didn't get to it until this weekend. This weekend I loaded the footage into Premiere Pro and found at least 10% of the clips are unrecognizable in any program (Windows Media Player, VLC, Media Encoder, MediaInfo, etc.). They all have varying amounts of data.

I DO have one SD card with the footage on it but I don't know for sure if it is the footage that was copied to the computer or if it was the 2nd card. That doesn't mean anything except that both cards had bad files created by the Sony camera, OR the one GOOD card had the data that I cleared off when this card appeared to copy fine and I can no longer compare files.

Any advice?

 

[NorBro]

Firstly, sorry to hear that...secondly, we've all been through it at some point and it's the same story, same advice; try different programs until something can recover something, even part of something.

Most of the time some of the footage is gone forever. Sometimes most of it if you're not having any success with multiple applications (especially highly-rated ones).

Final troubleshooting would be to send the card to a place to see what they could do (especially if you haven't formatted anything).

Final, final...if you can by any chance get the files to play in ANYTHING, record them. Like via an external recorder if they can playback on the camera, any camera...or screen recording on a computer.

 

[Doug Jensen]

Any advice?

Buy better cards and use a better workflow so problems are immediately identified and you know where each clip originated from.

 

[MrP]

Thanks for your replies. I appreciate them. NorBro, I think you're right that it's just trial and error. But I was hoping someone had come across something like this before. There are a lot of posts about recovering data but very few about files (especially a block of files) simply not being accessible. So far I haven't found anything that works It's just strange that it's a block of them.

Doug, can you give insight into a good workflow? I'm using Sony TOUGH cards from BH and have been using them for some time (and they are the 2nd revision cards), so I'm not sure what better cards you suggest. For the workflow, I always spot-check files and always, always do the file check to be sure they are identical from the card to the server. The spot check looked fine, but I must not have tried one of this group. Honestly, if dropping them into Premiere every single time I offload footage would be the workflow, I'll do it, I just never had this happen before. What's your workflow?

Thank you again,
-Stephen

 

[markfpv]

Not sure I'm following completely... do you have both the media cards (not reformatted yet) used and the A7sIII they were shot on? Have you tried putting a card into camera to see if the files play? Or if they camera tells you they need to be "restored" - or other similar message??

 

[NorBro]

In general, this should rarely happen if you use any decent cards. A fluke occurrence, if you will. But sometimes cameras might also break for a second - even with the best media - because they are tools, computers that aren't perfect.

It's hit or miss with which recovery software may work or may not, every situation is different.

And you can use software to check files but at the end of the day the software is just software and not a God or a super spirit who can always make sure every single pixel in your footage is perfect, you know?

The best thing you can do is record in as many ways as possible - when possible - and then check the footage as quickly as you can. And if you can't dual or triple record then when you transfer the footage, archive the card (physically) or archive the entire file structure into a folder and keep that copy of that folder in 2-3 places.

There is not much else...record in multiple ways, have a human review and check the footage as soon as possible, and keep what you recorded backed-up in its original state.

 

[MrP]

Not sure I'm following completely... do you have both the media cards (not reformatted yet) used and the A7sIII they were shot on? Have you tried putting a card into camera to see if the files play? Or if they camera tells you they need to be "restored" - or other similar message??

To be honest I didn't even think about putting the card back into the camera... I have 1 of the 2 cards unformatted. A selection of files on my server is bad and the same selection of files on the card are bad, but since I don't have both original cards I don't know if these files were originally from this card or from the other formatted card. In other words, I don't know if the files were corrupt on BOTH cards or if I just happened to copy the footage from the bad card, think all was well, then clear the good card only to find the files are bad and copying again from the presumably bad card does no good. Now I'm making myself confused....

I'll put the card back in the camera and see what happens! Not sure why I didn't think of it except that I was trying to troubleshoot over the busy weekend.

 

[Doug Jensen]

Doug, can you give insight into a good workflow? I'm using Sony TOUGH cards from BH and have been using them for some time (and they are the 2nd revision cards), so I'm not sure what better cards you suggest.

Stephen, you're already using excellent cards (I also use Sony TOUGH cards), and you purchased them from a reputable reseller, so let's assume the media itself is not the culprit.

You're probably doing most of things anyway, but here's my workflow:

I reformat all cards before I leave home and never reformat anything in the field.
I always format the cards onboard the camera that I will actually use for recording.
If the camera gives me the option of Quick Format or Full Format, I always choose Full.
I never eject/insert a card unless the camera is powered down.
I never power down the camera unless I'm sure the last file has finished writing.
I never let the battery run all the way down.
I never let the memory card reach full capacity.
I never use relay recording.
I never use the dual-record capability of a camera.
I never use an external recorder for backup.

I use a high-quality stand-alone card reader and never use the camera for ingest.
I ingest 100% of the contents of the memory card (all sub-folders and files), from the root folder on down.
I never extract just the video files alone.
I ingest by dragging and dropping the root folder on the memory card to a custom-named folder on an external HDD for archiving. This is archive A.
I then drag and drop from the card to a second external HDD. This is archive B.
I never make the B copy from the A copy.
If this is a current project that I need to edit right away, I then drag and drop from the card to one of my fast external SSDs that I use for editing.
If I have a lot of footage to ingest I will use ShotPut Pro to ingest to all three of those destinations at once. But usually I am not in a rush.
I launch Catalyst Browse and spot-check the ingested footage, making sure all thumbnails appear and that the clip numbering is continuous.
After ingest, I will set the memory card(s) aside and only reformat them when I am actually ready to use them again and know which camera they will be used with.

I've been shooting tapeless with Sony cameras since 2006. That's coming up on 17 years, and I have never had a corrupted card, a missing file, or even so much as a lost frame. 100% perfect track record with more than a dozen different cameras.

 

[Andy9]

Workflow additional: For a long time I’ve been using free software called Parashoot that does a simple (for me, final) file name / file size check and then, if the footage appears to be there at the destination, it will offer to do a “fake format” - it reverses all the ones and zeroes of the file header on the card. This is instantly reversible by Parashoot at any time. If you put the card in any camera, the camera will require you to format the card. I use it in conjunction with Hedge (it’s integrated, although you can also use it standalone).

https://ottomatic.io/o/parashoot/

It’s just one last line of defence - when I’ve ‘formatted’ the card, I haven’t actually formatted it.

Doug, why don’t you dual record? Surely probability loves a dual recorder.

 

[NorBro]

Doug does a little dual-recording but in the form of proxies...a few months ago he said he had one card in the camera with tons of clips from different shoots and loves how he can have access to the footage if he ever has to hand over the main media and doesn't have to rely on anyone sending the footage to him (I remember that because I thought it was a good idea).

 

[drboffa]

FWIW, Sony SD cards are the only cards that ever failed on me. They had at least one bad batch of cards a few years ago (the problem was widespread enough that they were forced to admit the problem and replace cards that were sent in).

I'm sure it's very uncommon, but the experience was bad enough that I've stopped trusting Sony cards since then.

In the instance of the corrupted Sony card, I was able to recover some footage via a recovery program. I don't recall exactly which one, but if you get desperate I could do some digging to find it.

Coincidentally, my experience was the same as yours (just with a GH5s): I copied my files to backup drives, erased one SD card to put it back in rotation and kept one SD card as a third backup. But the backup SD card (I almost always shoot dual card) was the corrupted one. It had all the files, and the file sizes were appropriate for video, but they were unplayable.

 

[Tom Roper]

Disk Drill worked for me and was cheap.

 

[Doug Jensen]

Doug, why don’t you dual record? Surely probability loves a dual recorder.

Don't need it. I have complete confidence in my cameras, my choice of media, and my workflow.
BTW, I don't wear a helmet when I ride in a car, and I only use a belt (no suspenders) to keep my pants up.

 

[Doug Jensen]

Doug does a little dual-recording but in the form of proxies...a few months ago he said he had one card in the camera with tons of clips from different shoots and loves how he can have access to the footage if he ever has to hand over the main media and doesn't have to rely on anyone sending the footage to him (I remember that because I thought it was a good idea).

Hey, you're paying attention better than I thought.
But I only record proxies on my Z750. Everything else I shoot is one copy only. And I have never had to use the proxies because the SxS cards are infallible.

You realize I've just jinxed myself, right?

 

[NorBro]

One time in middle school I won an award for best notes because I was so sickly neurotic about paying attention and writing down everything perfectly, even random lines which I thought had a purpose but were just chalk marks not completely erased from the previous day.

 

[Doug Jensen]

One time in middle school I won an award for best notes because I was so sickly neurotic about paying attention and writing down everything perfectly, even random lines which I thought had a purpose but were just chalk marks not completely erased from the previous day.

Rainman.

 

[cyvideo]

Are there any other solutions out there that can "fix" .mp4 files? They all have data (some are 3+GB) but none are playable. The files are on the PC. I don't need to recover deleted clips but rather somehow "fix" these...

I don't know about fix, but just had a similar problem to sort out. A cameraman came to me only yesterday, Monday with an XQD card with a UHD 50p concert shoot he shot on Sunday where things went horribly wrong. He was running a normal 14.8v V-Loc battery using D-Tap step-up DC-DC converter cable to step up to 19.5 volts for his FX6 when everything shut down. The camera wouldn't restart. Changed battery and still no restart. Swapped out the V-Loc for an onboard battery, it then fired up again and all went well. On closer inspection, it turned out the DC-DC converter cable had fried itself internally.

After the event the XQD was loaded back into the camera which prompted to run a "card restore" or whatever it's called. That failed with an "unrecognized media" media message.

This is what I did on a PC. I mounted the XQD in a card adapter and the PC saw the card okay. I found that the old Media Player Home Cinema would actually play the MXF files, but with no progress marker running on the timeline counter. The play indicator wasn't progressing along the play bar, but the file was playing. Every now and again, a bunch of corrupted pixelated frames would flash through as it played. Nothing else would even look at or mount this file. So even if you can't play any of the files, the following might work to help you out.

First step.
The step of last resort. I fired up "Shutter Encoder". In Shutter Encoder, I could drag the file into its window. I tried a "Rewrap" to MOV as this has got me out of problems in the past. This failed. So with nothing to lose, I selected "Create an image sequence from video". I selected JPG as the PNG file exports work out about 5-6MB per frame. The JPGs are around 0.5MB per frame. I then ran this export sequence. A while later, takes quite a while, I had a folder with thousands of JPG files. No audio, of course!

Next step.
As I stated earlier, the file would actually play in Media Player Home Cinema (MPC-HC v1.9.1x64) with sound. I captured the sound using Audacity using its Windows WASPI "record what you hear" option. Saved that out as a 48Hz 24-bit .WAV file.

Last step.
Brought all the still images into Resolve as an Image Sequence. Brought the captured audio WAV file into Resolve. Synced them up on the timeline and exported a new UHD 50p file.

File rescued! Lucky they had a couple of other cameras covering the event as that allowed them to cut around the corrupted frames in the reconstructed video. All those corrupted frames turned up as visually corrupted stills in the sequence.

Chris Young

Every editor should have this tool set!

https://www.shutterencoder.com/en/

 

[MrP]

Thanks so much for all of your thoughts and suggestions. For fear that I was getting obsessed I've given up for now... I was shooting with two cameras and the only shots I'll miss are ones that I remember getting but have similar shots on the other camera. So it's not the end of the world. But it's certainly frustrating that I don't know WHY it happened.

I think I've been doing what a lot of you have suggested but there are some better workflow options you suggested. I never thought about using Catalyst Browse to see thumbnails. Some footage is 10bit which Windows sometimes struggles to show thumbnails for- I bet Catalyst would work great for this. I love the idea of Parashoot but I'm on PC. I've always used Handbrake and XMedia Recode but never used Shutter Encoder- I've added that to my arsenal now. Thanks!

I tried the card in the camera (I have the camera custom name the cards so I know which camera the cards come from- CAM A or CAM B) and the camera didn't recognize any footage at all. Not even the good footage. No option to recover or restore. So, I assume the card is damaged (or at least the file structure is as the camera doesn't know there are playable files). Since I don't think there's much else to try with the card I tried recording a sample clip and it does show up in the camera and is playable. Just out of curiosity I played around with some of the XML files on the card (I made copies) to see if I could "add" the files that I know were good back into the camera's folder. But it's some trickery that I don't understand.

I would have more confidence if it was just recovering lost files (I have Easeus and someone else mentioned Disk Drill), but the files aren't lost, they are just corrupt.

In the past I had proxy recording turned on and even thought to check there incase some low-quality files were still accessible, but that was turned off, presumably when I was shooting at a higher quality or dual recording or some other setting that did not allow for proxies.

Chris, thanks so much for your thorough walkthrough! Unfortunately, I can't get the files to be read by any software. I had forgotten about Media Player Home Cinema as I've used VLC lately. I tried it with hope in my eyes but no luck I gave Shutter Encoder a try but it also gave an error once I clicked Encode (or whatever the "go" button is).

I'll let you know if I ever get these files but I have a tendency to get obsessed with something when it presents itself as a challenge... Sometimes that's good and sometimes it kills me. I think I need to give up... unless someone has more ideas!

Thank you again, everyone, I really appreciate your input!

-Stephen

 

[cyvideo]

I'll let you know if I ever get these files but I have a tendency to get obsessed with something when it presents itself as a challenge... Sometimes that's good and sometimes it kills me. I think I need to give up... unless someone has more ideas!

Last long shot, I guess. Can you upload one of the smaller files and post a link where some of us could download it and have a go at it? Dropbox, Google Drive, WeTransfer etc. If you do that also supply a known good ten-second file from the same camera if possible. Used for file reconstruction by some software. Understand your reluctance at giving up. Over the years I have acquired some weird software and apps, some of which have given surprising results in recovering and fixing things. One is a piece of software from a programmer guy in India. Never been commercially available, but surprisingly good. Always up for a challenge. Hate being beaten by technology!

Chris Young

 

[MrP]

Last long shot, I guess. Can you upload one of the smaller files and post a link where some of us could download it and have a go at it? Dropbox, Google Drive, WeTransfer etc. If you do that also supply a known good ten-second file from the same camera if possible. Used for file reconstruction by some software. Understand your reluctance at giving up. Over the years I have acquired some weird software and apps, some of which have given surprising results in recovering and fixing things. One is a piece of software from a programmer guy in India. Never been commercially available, but surprisingly good. Always up for a challenge. Hate being beaten by technology!

Chris Young

I'll take you up on that! I'll upload one and then I'll see if I can grab a few different versions of good files since I don't know for certain what framerate they would have been recorded in. On this particular project, we filmed in a few different frame rates (23.976 and 59.94). I think everything from this timeframe was just 8bit rather than 10bit which we used in specific challenging locations for daylight. This project is not a corporate or commercial project so I am happy to share a few clips.

Thank you again, Chris, I appreciate it.