Laptop offloading -- double time

Barry_Green

Moderator
Latest workflow innovation to report. Tonight we shot a commercial that required lots of interviews/testimonials and, consequently lots of P2 offloads. (well, not LOTS, only about five.)

We delivered the footage to the client on a hard disk, as always.

My plan was to copy the cards twice; once to the external hard disk for the client's master, and also onto the laptop's internal hard disk for safety, just in case. Redundancy is good, right? Even if it takes twice as long.

Well, I'd copy it to the external, and wait four minutes until it was done. Then I'd copy it to the internal drive, waiting that four minutes. By then it was almost time to swap out the next card. Which is all fine and dandy, but it'd be nice if there was a little more time available, a little more breathing room...

... and I got to thinking... y'know, that card is supposed to have an 80 megabyte per second data transfer rate. The hard disk is the bottleneck; it's only about 30 megabytes per second... which means there's a lot of "read" capability going unused... and there's no read/write head that needs to get moved around, because it's solid-state storage, so there's no wasted time in seek overhead...

... so I tried copying the next card to the external hard disk, and while it was copying I opened up a second window and dragged the card's contents onto the internal hard disk, just to see what type of speed hit it'd take. And the answer is: hardly any! Copying the card to both the internal and the external drives at the same time resulted in total transfer rates about identical to just copying it to the external drive alone. If it was any slower, it was barely slower!

I thought that was pretty cool. Don't know why I hadn't tried it before. This means you should probably be able to copy to two external hard disks simultaneously as well -- one on firewire, one on USB2 for example, so there's no bus speed conflicts.

Just thought I'd throw that out there, for those doing P2 offloads on a laptop -- you can get a second/redundant copy done in almost the same amount of time as it would otherwise take to just offload the card in the first place!
 
Nice!

Any advice that saves time is greatly appreciated.

One thing I really like about the 4gig P2 cards is that they are the perfect size for a DVD-R hard copy to be made. For me, it is very important to archive the info as soon as possible. The 4gig P2 cards by their very nature reduce the need for adjustments to fit on a 4.7 (4.2) DVD-R.
 
Maybe a little warning: I did that (double copying data same time) some years ago with Apple to a scci dual UW raid and to an internal drive at the same time, and endid up with some files being corrupted. Don't know why. So if you do this for redundancy you need to know for sure this is -always- a save way.
 
Barry that is good news indeed. I was going to try that same thing. Have you encountered any problems?


I get my p2store today! Looking forward to being able to get a few hours of shooting together before having to back up. Shoot, I might even tackel some 1080 footage now... besides the timelapse and interval shooting. that's not too hoggy at 1080 at all.
 
I tested the files after copying and all was well. I can understand xray's caution when doing this from a hard disk source, as it would probably mean a whole lot of seeking and read-heads jumping around, but the difference is: this is a solid-state source. There is no additional stress being placed on the card. I just can't think of any reason why it would be any less reliable whatsoever.

Only thing I'd say is that you want to make sure that the devices you're copying to are on separate busses -- like, don't copy to two USB2 drives, because that'll clog your bandwidth. Copy to the internal drive and an external drive, and that keeps the traffic on separate busses. Or copy to an external firewire drive and an external USB drive, which is still separate busses.
 
Barry_Green said:
Latest workflow innovation to report. Tonight we shot a commercial that required lots of interviews/testimonials and, consequently lots of P2 offloads. (well, not LOTS, only about five.)

We delivered the footage to the client on a hard disk, as always.

My plan was to copy the cards twice; once to the external hard disk for the client's master, and also onto the laptop's internal hard disk for safety, just in case. Redundancy is good, right? Even if it takes twice as long.

Well, I'd copy it to the external, and wait four minutes until it was done. Then I'd copy it to the internal drive, waiting that four minutes. By then it was almost time to swap out the next card. Which is all fine and dandy, but it'd be nice if there was a little more time available, a little more breathing room...

... and I got to thinking... y'know, that card is supposed to have an 80 megabyte per second data transfer rate. The hard disk is the bottleneck; it's only about 30 megabytes per second... which means there's a lot of "read" capability going unused... and there's no read/write head that needs to get moved around, because it's solid-state storage, so there's no wasted time in seek overhead...

... so I tried copying the next card to the external hard disk, and while it was copying I opened up a second window and dragged the card's contents onto the internal hard disk, just to see what type of speed hit it'd take. And the answer is: hardly any! Copying the card to both the internal and the external drives at the same time resulted in total transfer rates about identical to just copying it to the external drive alone. If it was any slower, it was barely slower!

I thought that was pretty cool. Don't know why I hadn't tried it before. This means you should probably be able to copy to two external hard disks simultaneously as well -- one on firewire, one on USB2 for example, so there's no bus speed conflicts.

Just thought I'd throw that out there, for those doing P2 offloads on a laptop -- you can get a second/redundant copy done in almost the same amount of time as it would otherwise take to just offload the card in the first place!


Now if someone could come up with a program where you could just drag the files to an icon, and it automatically goes to both drives, that would make it even easier.

On paper, that doesn't seem like a hard thing to do...
Maybe they could impliment it in the next version of the P2 viewer.
 
Easy.

Set up a Raid level 1 mirror in Apple disk Utility and you have Identical copies to each drive in the raid which can be split and passed to client and stored as backup.

The beauty of the Mac...
 
cool news. Can this be done for idential external drives or has to be two raided drives in an enclosure? I'm going to have to buy more drive space.. hopefully
 
This is a good trick. Thanks Barry. Also you can use the automator program in the mac OSX to do this. A folder actions based double drop kinda thing. Curruption errors are a concern but I think, not sure, that if a file gets currupted in transfer a warning window will pop up.
 
Great idea, Barry! Yesterday I had a 5 month old 250GB fail. Fortunately, there wasn't much on there that I needed. Just a word from the trenches that HDD's can and do fail at the most unexpected times. Always, always, always keep backup's or you Will_Get_Burned. This is particularly true if you work on Mac because of the lack of a comprehensive defrag utility in OS X. Case in point, this is my 4th HDD failure in 3 years.
 
pretopost said:
Easy.

Set up a Raid level 1 mirror in Apple disk Utility and you have Identical copies to each drive in the raid which can be split and passed to client and stored as backup.

The beauty of the Mac...

Can you give us more details on how to do this? I am new to Mac from the world of windows and can't even find half of these utilities!!

Help!
 
It's easy to set up. Use the Disk Utility appliction found in Applications/Utilities and select one of the two drives you want to make into a RAID Setup. Follow the prompts after that. It will ask what type of Raid choose Raid 1 as that is a mirroring/redundant setup. The app will then format both drives and they will appear as one volume on the desktop but in reality are two identical drives with their own directories. If there is a hardware failure than you have a complete back up of the other drive.

This can be done with both internal and external firewire drives. My setup is two Firewire 800 Drives that are 500 GB a piece. I have had two drives go down in the last 2 years and this has saved my butt a few times...
 
thisiswells said:
That's funny. "Mac, the only operating system that does not include basic disk utilities." Defragmenting sells for $30 and disk partitions will cost you $45! Gotta love that!

http://www.coriolis-systems.com/iDefrag.php

Not true. Partitions can be done with Disk Utility Program that is included, including the above Raid setup--tell me a PC that has RAID software for free out of the box.

Disk defragging can be done with Tech Tools pro which is included with AppleCare protection which IMO is a must if you run a business.
 
Disk defragging can be done with Tech Tools pro which is included with AppleCare protection which IMO is a must if you run a business.
I have AppleCare, tell me about it. I'm not all that IT savvy.
 
pretopost said:
--tell me a PC that has RAID software for free out of the box.

Um....any PC with Windows XP can do this easily. At least Raid 1 which is mirroring and raid 0 which is striping. That is what we are talking about. Can the Mac do Raid 5? That takes at least 3 disks and 1 disk holds parity data to rebuild the others in the event of 1 disk failing. Windows XP does not do Raid 5. It however easily handles Raid 0 and 1.
 
icicle22 said:
Um....any PC with Windows XP can do this easily. At least Raid 1 which is mirroring and raid 0 which is striping. That is what we are talking about. Can the Mac do Raid 5? That takes at least 3 disks and 1 disk holds parity data to rebuild the others in the event of 1 disk failing. Windows XP does not do Raid 5. It however easily handles Raid 0 and 1.

No just Raid 0 or 1. Sorry bout the PC stab. You learn something new every day. For Raid 5 you'll need another solution like a Promax setup or Xserve Raid.
 
thisiswells said:
I have AppleCare, tell me about it. I'm not all that IT savvy.

There's a disk that was inside your apple Care. It has disk defragging and diagnosing on it. It's called Tech Tools. They have an upgrade for purchase.
 
right click, copy, and paste 1 paste 2... you can make drives icons you can past ontop of. thats all baby.
 
Back
Top