Best practice of backing up original file footage

[Hywel]

Checksums is a very good point. Although I don't particularly want to work with disk images, I really ought to switch to something which uses checksum on the file copies. I can probably write a little rsync shell script to do it on file copies, but checking the whole lot with one checksum is a very good thing

 

[Hywel]

The Drobo looks like a great system. Is that what you use for your Mac pro raid array? I'm looking into setting up a different system than what I'm using now. The drobo looks very versatile.

I have a total of three Drobo systems, and an Apple RAID card with 3 x 1 TB disks in RAID5 on my MacPro.

I edit footage from the Apple RAID array- it is fast and robust and directly connected by a high speed connection.

I have two DroboPro's set up as my file archive, but copy stuff to the Apple RAID array on the MacPro to edit. I have them connected via Gigabit Ethernet and they are fast enough to edit from, but not quite as fast as the built in array. So while I have edited direct from them, I usually copy across to the MacPro built in array first.

I also have a Drobo connected via Firewire, which I use purely for Time Machine backups. I wouldn't want to edit from that, it is definitely slower than the others, but is fine for automated backups.

Cheers, Hywel.

 

[Rick Burnett]

Hmmm, think I need to dust off my AppleScript book, that is a really good idea to setup a method of archiving. Thanks for that!

 

[djkarn105]

Hmmm, think I need to dust off my AppleScript book, that is a really good idea to setup a method of archiving. Thanks for that!

Yeah, I've been meaning to polish it up and release it to the boards but the past few months have been hell for me at work so I haven't had time. Also, I should add the disclaimer that I've not tested it with AF100 footage but assuming it works the same as GH1 footage should not be a problem.

 

[Rick Burnett]

Well if you do share, share here as it might make a great starting point for others as well

 

[Mike@AF]

Checksums is a very good point. Although I don't particularly want to work with disk images, I really ought to switch to something which uses checksum on the file copies. I can probably write a little rsync shell script to do it on file copies, but checking the whole lot with one checksum is a very good thing

Same here. With a disk image you can lose all the files in the image if it gets corrupted. ShotPut Pro will do the checksum/verify and will copy to 2 drives at the same time faster than Finder will.

I do what Hywel does. Backup the card to 2 portable drives. When I edit I also have a copy of the cards on a RAID-5 array. I use a OWC Mercury Elite-AL Pro QX2 8TB RAID-5 array for editing. It gives you USB, Firewire, and eSATA. I typically use eSATA for editing, but a lot of the time FW800 works well also.

 

[djkarn105]

Same here. With a disk image you can lose all the files in the image if it gets corrupted. ShotPut Pro will do the checksum/verify and will copy to 2 drives at the same time faster than Finder will.

I don't see why this would be an issue if you have redundant copies and are error checking them? And there are utilities out there that will do all this stuff for you but they're not free.

I can probably write a little rsync shell script to do it on file copies, but checking the whole lot with one checksum is a very good thing

When I first started doing this I was using rsync too. The problem is rsync runs the checksum as it transfers whereas with the disk image the checksum is built in so after you've built it to the first drive, you can copy it to the second drive and run the checksum much faster.

 

[Rick Burnett]

The problem is this, in the worst case scenario if you find one drive has gone bad and the disk image has a bad spot on it, can you recovered partial files out of the disk image?

If you are shooting on a 32G or 64G card, you might have A LOT of footage in that archive. I am not saying the practice is good or bad, just that the question becomes when you get a bad sector on the file, can anything be recovered out of it in the change it becomes the only copy? I think that is a valid concern.

I don't use disk images for one main reason, I want to be able to see what is in the directory quickly without mounting as I tend to have LOTS of card copies. I want to be able to search quick through multiple projects sometimes when I want to look for something, and I like to keep them original, and in their native format.

 

[Andrew J. Allsbury]

For those on a PC with an extra PC laying around...Windows Home Server is an awesome option.

 

[Stu Siegal]

Checksums is a very good point. Although I don't particularly want to work with disk images, I really ought to switch to something which uses checksum on the file copies. I can probably write a little rsync shell script to do it on file copies, but checking the whole lot with one checksum is a very good thing

Carbon Copy Cloner does a checksum and avoids a disc image.

 

[Mike@AF]

I don't see why this would be an issue if you have redundant copies and are error checking them? And there are utilities out there that will do all this stuff for you but they're not free.

Ease of access? You have to mount a disk image to open a file in it or search the contents. If you're dealing with a project that has numerous reels you'd have to spend the time to mount all those disk images, then search them, get what you want, and then unmount them. It's just not as easy or practical when programs to do the checksum and copy (and at a faster speed) aren't really that expensive to begin with. Sure, they're not free, but they give you security AND convenience, which a disk image doesn't.

 

[djkarn105]

Ease of access? You have to mount a disk image to open a file in it or search the contents. If you're dealing with a project that has numerous reels you'd have to spend the time to mount all those disk images, then search them, get what you want, and then unmount them. It's just not as easy or practical when programs to do the checksum and copy (and at a faster speed) aren't really that expensive to begin with. Sure, they're not free, but they give you security AND convenience, which a disk image doesn't.

A good point, I guess for my workflow, I transcode the footage to prores and only go back to the archives when I re-import for online so it's not an issue.

 

[Rick Burnett]

AVVCAM utility is a no go so far if you did not import from the SD through the utility. Tried to fool it, but it wasn't having it!

I see that Toast 10 can play MTS files, does it have a viewer that can scan through from a starting point? I don't have toast 10.

 

[tflak]

Toast 10 has a pretty good stand-alone viewer that's hidden in the package (if I remember right). It can scan from any point and also displays stream info. It can also transcode MTS files to Pro Res without needing the PRIVATE structure. I seldom use it for that, preferring log and transfer. Biggest drawback to using Toast for transcoding is there's no way to save clip points for a transcoding session. So, if you've selected multiple parts from a single clip and then need to re-ingest that info isn't there. Or even single trim points for that matter.

Transcode quality generally seems pretty good. But I did have a clip with lots of fine detail; close up of goldenrod blowing in the wind with honeybees, and Toast introduced some unexpected noise. Nothing too bad, just surprised to see it. Unfortunately, I couldn't transcode with FCP to compare since I only had the MTS file. Which is the only reason I used Toast in the first place.