BobDiaz
12-17-2007, 11:21 AM
Both the HVX-200 and the EX-1 users are going to face the issue of how to store their videos for the long term. Fortunately, this issue has come up before with large networks and the possible solutions are well known. The perfect solution doesn't exist, so users will need to decide which solution is best for their needs. This technical note addresses the choices and issues you will need to face when selecting your backup plan.
Removable Hard Drives:
Placing all your critical video files on a single hard drive is just plain stupid. Sooner of later the drive will fail and all your hard work will be gone. There are companies that do offer a recovery service for failed hard drives, but a better and less expensive way is found below.
The solution is to back up your videos onto 2, or better yet, 3 removable hard drives. If one drive fails, you can recover the data from the other drive(s). If you really want to be safe, store the other drive or drives at a different location or locations. The reduces the risk of a fire, flood, or some other unexpected event from destroying all your drives.
Another thing to consider is using multiple removable drives from different manufacturers. While the chance of purchasing a product with a manufacturing design flaw is small, getting product from different manufacturers reduces this possibility to near zero for every set of drives. So your first set of drives is from company "A" and your second copy is stored on drives from company "B". If you use a third copy, that would come from company "C".
Unfortunately, putting drives on a shelf for years at a time, without running them, may result in all the drives freezing up. This means that you'll need to go to all the drives to run a file check program once or twice a year to avoid this. As soon as one drive fails the file check program, you'll need to get new drives and make copies of the files from the good drive(s).
It's difficult to say exactly when one of your removable backup drives will fail, because they are only plugged in for backup and file verification. Three to five years is the normal life for a drive that receives regular use, but maybe with limited use, a drive like this may survive 7 years. However, a dropped drive is most likely dead as soon as it hits the floor.
Prices of USB and Firewire removable hard drives have dropped over the years and will continue to drop. As of today, the Mac Mall catalog shows a 2TB (2,000GB) My Book Pro Edition II selling for $639. Such a drive would hold over 100 hours of 35Mb/s material. That comes to under $6.40 per hour of material. The 500GB USB drive by LaCie sells for $118. Such a drive would hold over 25 hours of 35Mb/s material at under $5 per hour.
Just multiply the $6.40 or $5 by 2 or 3 to see what each hour of material would cost when placed on 2 or 3 drives and cost ranges from a low of $10 to high of under $20 per hour of material. Remember this covers you for a range of 3 to 7 years. After that, you'll need to get another set of drives to store the data. However, by then, the price per GB (or per hour) of material will be considerably less than today's price.
A sure bet is that in the future, even larger drives will appear at the same price as existing hard drives. Thus, in the future, when you move files to newer drives, you'll be able to move several "smaller drives" onto larger drives.
Optical:
In theory, optical media should have a life-span of 20 to 250 years.
http://www.techmount.com/index.php/20051205/dvd-lifespan/
Articles on CD and DVD Rot have placed a major shadow of doubt on whether optical media will last that long.
http://en.wikipedia.org/wiki/CD_rot
However, the correct answer depends on several factors that can't be judged by the average user.
http://www.larryjordan.biz/articles/lj_dvd_life.html
My suggestion here is, to stick to major name brand disks and if possible, select disks that say they are for archival and are of a higher quality. That may be a marketing trick, but it could also be a higher grade disk. The most important thing to do is to record the video files on several different brands of disks. Even in the worst case of all the disks getting rot, the chances of them starting to show rot at the same time is near zero. Remember you will need to test the disks every year by seeing if you can copy files into a system.
SL (Single Layer) DVDs are very inexpensive, but would only hold around 12 to 15 minutes of video per disk. DL (Double Layer) DVDs cost considerably more per GB, but could hold around 24 to 30 minutes of video. Because prices of the blank media vary so widely, the costs vary from about $1 to $6 per hour of material (one copy).
HD DVD would hold 15GB SL (>45 minutes) or 30GB DL (>90 minutes), but burners are just not practical yet. Blu-ray is a bit better with 25GB SL (>75 minutes) and 50GB DL (>150 minutes). The big question is whether either format will be around in the next 5 years?
There is a chance that both formats could die in the format war and who knows, maybe an even larger capacity optical media will take over. While many pretend to act like they know how the format war will end, chances are we won't know a clear cut winner or looser of this war until 2009 or 2010. By then, if one or both of the formats wins, it should be a reasonable long term solution.
With Blu-ray optical storage, SL 25GB disks sell for around $14 to $30 and DL 50GB disks sell for around $50 to $60. That comes to around $11 to $24 per hour for Blu-ray (one copy). However over time, the price for blank media will drop.
If you can find an HD DVD burner, blank SL 15GB Disks sell for around $12 to $15. That comes to around $16 to $25 per hour (one copy). However, over time, these prices will drop.
Back-Up Tape:
This seems somewhat odd to talk about "going tapeless" by copying all your video data to tape. However, backup tape systems can store large amounts of data rather cheaply. Depending on who's system you chose, the price can range from under $2 to over $5 per hour of video (one copy).
Some warnings on tape are needed; ALWAYS VERIFY ANY DATA YOU WRITE TO TAPE!!!!!! There are horror stories of people who backed up the server data every night onto tape, but failed to verify the data after it was written. It turns out that the tape drive went through all the motions, but recorded garbage to the tape. It was only after a major hard drive crash that the person discovered that all the backups were worthless.
Also, the estimate of tape's life-span is about 10 to 30 years. The worst problem is that tape drives change so quickly, that unless you have 2 or better yet 3 identical tape drives, you might not be able to read the tapes when the drive dies. Plus, getting a replacement drive that can read the tape may be next to impossible.
Storage While Out In The Field:
Storing your files to a laptop's hard drive seems reasonable, BUT I wouldn't want to have the drive of the laptop be my only storage location. One drop could quickly loose weeks of work. Having a second or third copy on a USB Powered drive(s) would provide additional protection.
One interesting idea I read on one of the forums was to use the 8GB, 16GB, or 32GB USB Flash Drives to back up your files while in the field. You can now find 8GB Flash drives for under $100 and 16GB Flash drives for under $200. The 32GB Flash drives sell for around $300 to $500. You can expect that over the years, these prices will drop.
Compared to the cost of the SxS cards, these flash drives are only a fraction of the cost. They are not as fast as the SxS cards, but they may be fast enough to copy all the files during the breaks between shooting.
Compared to the cost of a USB Powered Hard Drive, the USB Flash Drives cost about 10x more per GB. Unlike hard drives, USB Flash Drives can be dropped without any real problem. Anyone one going with USB Flash Drives is paying more to have the extra protection. Still, I never like to work with just a single copy of my files. So, even with Flash Drive Storage, having a second backup on a Hard Drive is still a good idea.
Bob Diaz
Removable Hard Drives:
Placing all your critical video files on a single hard drive is just plain stupid. Sooner of later the drive will fail and all your hard work will be gone. There are companies that do offer a recovery service for failed hard drives, but a better and less expensive way is found below.
The solution is to back up your videos onto 2, or better yet, 3 removable hard drives. If one drive fails, you can recover the data from the other drive(s). If you really want to be safe, store the other drive or drives at a different location or locations. The reduces the risk of a fire, flood, or some other unexpected event from destroying all your drives.
Another thing to consider is using multiple removable drives from different manufacturers. While the chance of purchasing a product with a manufacturing design flaw is small, getting product from different manufacturers reduces this possibility to near zero for every set of drives. So your first set of drives is from company "A" and your second copy is stored on drives from company "B". If you use a third copy, that would come from company "C".
Unfortunately, putting drives on a shelf for years at a time, without running them, may result in all the drives freezing up. This means that you'll need to go to all the drives to run a file check program once or twice a year to avoid this. As soon as one drive fails the file check program, you'll need to get new drives and make copies of the files from the good drive(s).
It's difficult to say exactly when one of your removable backup drives will fail, because they are only plugged in for backup and file verification. Three to five years is the normal life for a drive that receives regular use, but maybe with limited use, a drive like this may survive 7 years. However, a dropped drive is most likely dead as soon as it hits the floor.
Prices of USB and Firewire removable hard drives have dropped over the years and will continue to drop. As of today, the Mac Mall catalog shows a 2TB (2,000GB) My Book Pro Edition II selling for $639. Such a drive would hold over 100 hours of 35Mb/s material. That comes to under $6.40 per hour of material. The 500GB USB drive by LaCie sells for $118. Such a drive would hold over 25 hours of 35Mb/s material at under $5 per hour.
Just multiply the $6.40 or $5 by 2 or 3 to see what each hour of material would cost when placed on 2 or 3 drives and cost ranges from a low of $10 to high of under $20 per hour of material. Remember this covers you for a range of 3 to 7 years. After that, you'll need to get another set of drives to store the data. However, by then, the price per GB (or per hour) of material will be considerably less than today's price.
A sure bet is that in the future, even larger drives will appear at the same price as existing hard drives. Thus, in the future, when you move files to newer drives, you'll be able to move several "smaller drives" onto larger drives.
Optical:
In theory, optical media should have a life-span of 20 to 250 years.
http://www.techmount.com/index.php/20051205/dvd-lifespan/
Articles on CD and DVD Rot have placed a major shadow of doubt on whether optical media will last that long.
http://en.wikipedia.org/wiki/CD_rot
However, the correct answer depends on several factors that can't be judged by the average user.
http://www.larryjordan.biz/articles/lj_dvd_life.html
My suggestion here is, to stick to major name brand disks and if possible, select disks that say they are for archival and are of a higher quality. That may be a marketing trick, but it could also be a higher grade disk. The most important thing to do is to record the video files on several different brands of disks. Even in the worst case of all the disks getting rot, the chances of them starting to show rot at the same time is near zero. Remember you will need to test the disks every year by seeing if you can copy files into a system.
SL (Single Layer) DVDs are very inexpensive, but would only hold around 12 to 15 minutes of video per disk. DL (Double Layer) DVDs cost considerably more per GB, but could hold around 24 to 30 minutes of video. Because prices of the blank media vary so widely, the costs vary from about $1 to $6 per hour of material (one copy).
HD DVD would hold 15GB SL (>45 minutes) or 30GB DL (>90 minutes), but burners are just not practical yet. Blu-ray is a bit better with 25GB SL (>75 minutes) and 50GB DL (>150 minutes). The big question is whether either format will be around in the next 5 years?
There is a chance that both formats could die in the format war and who knows, maybe an even larger capacity optical media will take over. While many pretend to act like they know how the format war will end, chances are we won't know a clear cut winner or looser of this war until 2009 or 2010. By then, if one or both of the formats wins, it should be a reasonable long term solution.
With Blu-ray optical storage, SL 25GB disks sell for around $14 to $30 and DL 50GB disks sell for around $50 to $60. That comes to around $11 to $24 per hour for Blu-ray (one copy). However over time, the price for blank media will drop.
If you can find an HD DVD burner, blank SL 15GB Disks sell for around $12 to $15. That comes to around $16 to $25 per hour (one copy). However, over time, these prices will drop.
Back-Up Tape:
This seems somewhat odd to talk about "going tapeless" by copying all your video data to tape. However, backup tape systems can store large amounts of data rather cheaply. Depending on who's system you chose, the price can range from under $2 to over $5 per hour of video (one copy).
Some warnings on tape are needed; ALWAYS VERIFY ANY DATA YOU WRITE TO TAPE!!!!!! There are horror stories of people who backed up the server data every night onto tape, but failed to verify the data after it was written. It turns out that the tape drive went through all the motions, but recorded garbage to the tape. It was only after a major hard drive crash that the person discovered that all the backups were worthless.
Also, the estimate of tape's life-span is about 10 to 30 years. The worst problem is that tape drives change so quickly, that unless you have 2 or better yet 3 identical tape drives, you might not be able to read the tapes when the drive dies. Plus, getting a replacement drive that can read the tape may be next to impossible.
Storage While Out In The Field:
Storing your files to a laptop's hard drive seems reasonable, BUT I wouldn't want to have the drive of the laptop be my only storage location. One drop could quickly loose weeks of work. Having a second or third copy on a USB Powered drive(s) would provide additional protection.
One interesting idea I read on one of the forums was to use the 8GB, 16GB, or 32GB USB Flash Drives to back up your files while in the field. You can now find 8GB Flash drives for under $100 and 16GB Flash drives for under $200. The 32GB Flash drives sell for around $300 to $500. You can expect that over the years, these prices will drop.
Compared to the cost of the SxS cards, these flash drives are only a fraction of the cost. They are not as fast as the SxS cards, but they may be fast enough to copy all the files during the breaks between shooting.
Compared to the cost of a USB Powered Hard Drive, the USB Flash Drives cost about 10x more per GB. Unlike hard drives, USB Flash Drives can be dropped without any real problem. Anyone one going with USB Flash Drives is paying more to have the extra protection. Still, I never like to work with just a single copy of my files. So, even with Flash Drive Storage, having a second backup on a Hard Drive is still a good idea.
Bob Diaz