BobDiaz
12-30-2005, 10:00 PM
If you think about it, anything you record on a P2 card is really computer data and if what you've recorded is important, you'll want to keep it. The problem of safely storing data is not new, both PC and MAC users have had to deal with this issue for roughly 2 decades. The solutions that exist for a PC and a MAC are the same solutions we can consider for P2 data.
As with any backup system, if you want to be safe, you need to have a second copy stored at ANOTHER LOCATION. If your current collection of video tapes does not have a second copy at another location, you are at risk for losing all or some your videos. A fire, flood, explosion, or some other disaster could wipe it all out. Another possibility is that on the day you need to recover archive footage, the tape deck decides to "eat" the tape. I know the risk of this happening is small, but it's a case of if this happens, what is it going to cost you?
As for long term storage of data, we will assume that proper storage conditions are observed. Extremes in temperature and humidity are avoided. Likewise, the media is kept from sunlight, dust, and chemicals that would reduce the life of the media.
TAPE:
Tapes are very popular for backing up hard drives. There is a wide range of options for tape backup systems. Prices vary from a few hundred to a few thousand dollars for the tape drives. In terms of cost, tape costs vary from around $0.20 per GB to $1.00 per GB. The lifespan of the data on the tape is roughly 10 to 20 years. Tape is easily damaged, so care in handling is important.
What many people don't know is that when writing to a tape, a VERIFY after the write is VERY important. I personally know of one case and have heard of others where the person backed up the data to tape every day, but did NOT verify the data. When the system's hard drive crashed, all the backup tapes were unreadable. Had the person doing the backup did a verify after a write, they would have been able to catch the problem before the disaster.
REMOVABLE HARD DRIVES:
Removable Hard Drives make it easy to quickly access data on the drive. Once stored on the drive, the drive can be placed in a cabinet. The sizes of the drives vary, but the cost is around $0.60 to $1.00 per GB. The good news is that the price per GB can be expected to drop by around 60% of last year's cost every year. This means that in 2 years, the cost should be around $0.22 to $0.36 per GB. As for lifespan, more estimates assume that the drive is left running and underestimate the life of the data on the drive. Given that the Hard Drive is magnetic, like tape, 10 to 20 years or more for storage does not seem unreasonable.
Now the bad news, if you drop the drive and you may need to kiss your data goodbye. There are companies that can restore damaged hard drives, but it will be expensive and even then, some data may be lost. This is why you always want to have a second copy of the data in another location.
OPTICAL MEDIA:
Optical Media is expected to safely hold it's data from around 10 to 100 years. Some companies say their disks will maintain data for 200 years. The NIST (National Institute of Standards and Technology, USA) says, "Among the manufacturers that have done testing, there is consensus that, under recommended storage conditions, CD-R, DVD-R, and DVD+R discs should have a life expectancy of 100 to 200 years or more..." HOWEVER, they do stress that more tests need to be performed. If you use the highest grade DVDs, expecting a data lifespan of 50 years or more is not unreasonable. (More on this issue later.)
See:
http://www.itl.nist.gov/div895/carefordisc/
As far as costs, the DL (Double Layer) 8.5GB disks sell for around $0.65 to $1.20 per GB, but the price is expected to drop over time. The smaller 4.7GB Disks sell for around $0.10 to $0.50 per GB.
To understand how much data is generated per hour and how many disks it will take to back it up, see the chart below. In this chart, I provide the //Number of 4.7 GB DVDs required // followed by the Number of 8.5 GB DVDs required.//
DVCPRO HD (at maximum data rate) = 60 GB pre Hour //15//8//Disks
DVCPRO50 or 720@30p = 30 GB per Hour //8//4//Disks
720@24p = 24 GB per Hour //6//3//Disks
DV = 15 GB per Hour //4//2//Disks
As you can see from the table above, at the maximum data rate, you need lots of disks, but one or two steps down and the number of disks required is not too bad. Because of the long time to burn DVDs, it would be wise to have a second computer to burn the disks. Just set up the first burn and come back some time later and change disks. It is somewhat pointless to stare at a machine waiting to change disks.
While the DL Disks cost more per GB right now, they may be the better choice because they require less disk changes. Once the price per GB of the DL Disks drops, these will be the disks of choice.
Some cautions about using DVDs for archival work:
FIRST, always use the highest grade DVDs. The really cheap stuff has been known to not last and some cheap disks delaminate after about 1 year. The link below will help you with what are the highest grade DVDs:
http://www.digitalfaq.com/media/dvdmedia.htm
SECOND, use the write once read many times DVDs (DVD-R, DVD+R). The DVD-RW and DVD-RAM will not hold the data as long as the first kind.
THIRD, do NOT use a solvent-based marker to write on the DVDs. The solvents may slowly work their way through the media and destroy the data on the disk. If you must mark the disk, use a NON-solvent-based marker and write only on the clear part in the center of the disk. Additional details can be found at:
http://www.itl.nist.gov/div895/carefordisc/disccare.html
THE FUTURE:
I can safely say that the price per GB of storage will drop every year. Also, new technologies will open new doors and generate new options. Blu-Ray single layer disks will hold 25 GB and the double layer disks will hold 50 GB. The Holographic Blue Laser disks will hold even more data (300GB) per disk. InPhase says that their disks will have a 50 year archival life and a media cost of $0.25 per GB.
http://www.inphase-technologies.com/index.html
http://www.enterprisestorageforum.com/technology/news/article.php/3523316
Even if 99% of the R&D for future storage is a failure, there are so many companies doing so much research into improved technologies, enough will come up with something even better than what I've listed so far.
SO WHAT SHOULD I USE FOR NOW?
The good news is we have many choices to select from. The bad news is deciding which is best for your needs. The correct answer depends on how much data you plan on generating per week, will the data need to survive past 10 or 20 years, how much are you willing to spend on the initial investment, and how much per GB will you be paying.
Another issue is technical obsolescence. Consider a tape backup system, you purchase a unit that works for several years. One day the drive fails and you discover that the company that made the tape drive is out of business or quit making the product; how will you recover the data on the tapes? You may find a used drive for sale OR maybe when you first purchased the drive, you purchased several of the same tape drives to be used on other systems. Another approach would be to put your second copy on a different media. Even if "xyz's" tape drive is no longer made, a second copy exists in another form.
To go into detail would take up too much space, but let me summarize some general guidelines:
TAPE: The price for the tape drive system varies and you get what you pay for. In general, the more expensive drives offer faster transfer rates, higher capacity, and most important of all, a more rugged drive. This can be important when you have massive amounts of data to backup every week. If the manufacturer publishes a MTBF (Mean Time Before Failure), this will give you some idea how long you can run the drive before you have a 50%/50% chance of a failure. Remember the real cost of this system is not just the drive, but you have to consider the cost of the media and how long it will take to do a transfer. Tape can be used in high and medium volume operations.
REMOVABLE HARD DRIVE: If it wasn't for the issue of killing a drive when you drop it, the removable hard drive would be perfect for medium and small operations. It can still be ideal if you have a second copy on another drive. (If you think about it, how often do you drop a video camera.) The random access nature of the drive makes it tempting even for large operations. Given how prices drop to 60% of last years price every year, this is a tempting option.
OPTICAL MEDIA: In a small operation with limited data every week, even 4.7GB and 8.5GB DVDs can be very practical. You will need a second computer to burn the disks and have to change the disks ever so often. After Blu-Ray Disks become practical and cost effective, this will make going optical for storage more attractive for everyone. If you record something that's VERY important and you want it to last, optical media is the best bet.
Last of all I leave you with one more link to look at. The article was written in 2004, but it is still worth reading:
http://asia.cnet.com/reviews/hardware/storage/0,39034915,39091141-1,00.htm
Bob Diaz
Wow that was a LONG post!!!
:beer:
As with any backup system, if you want to be safe, you need to have a second copy stored at ANOTHER LOCATION. If your current collection of video tapes does not have a second copy at another location, you are at risk for losing all or some your videos. A fire, flood, explosion, or some other disaster could wipe it all out. Another possibility is that on the day you need to recover archive footage, the tape deck decides to "eat" the tape. I know the risk of this happening is small, but it's a case of if this happens, what is it going to cost you?
As for long term storage of data, we will assume that proper storage conditions are observed. Extremes in temperature and humidity are avoided. Likewise, the media is kept from sunlight, dust, and chemicals that would reduce the life of the media.
TAPE:
Tapes are very popular for backing up hard drives. There is a wide range of options for tape backup systems. Prices vary from a few hundred to a few thousand dollars for the tape drives. In terms of cost, tape costs vary from around $0.20 per GB to $1.00 per GB. The lifespan of the data on the tape is roughly 10 to 20 years. Tape is easily damaged, so care in handling is important.
What many people don't know is that when writing to a tape, a VERIFY after the write is VERY important. I personally know of one case and have heard of others where the person backed up the data to tape every day, but did NOT verify the data. When the system's hard drive crashed, all the backup tapes were unreadable. Had the person doing the backup did a verify after a write, they would have been able to catch the problem before the disaster.
REMOVABLE HARD DRIVES:
Removable Hard Drives make it easy to quickly access data on the drive. Once stored on the drive, the drive can be placed in a cabinet. The sizes of the drives vary, but the cost is around $0.60 to $1.00 per GB. The good news is that the price per GB can be expected to drop by around 60% of last year's cost every year. This means that in 2 years, the cost should be around $0.22 to $0.36 per GB. As for lifespan, more estimates assume that the drive is left running and underestimate the life of the data on the drive. Given that the Hard Drive is magnetic, like tape, 10 to 20 years or more for storage does not seem unreasonable.
Now the bad news, if you drop the drive and you may need to kiss your data goodbye. There are companies that can restore damaged hard drives, but it will be expensive and even then, some data may be lost. This is why you always want to have a second copy of the data in another location.
OPTICAL MEDIA:
Optical Media is expected to safely hold it's data from around 10 to 100 years. Some companies say their disks will maintain data for 200 years. The NIST (National Institute of Standards and Technology, USA) says, "Among the manufacturers that have done testing, there is consensus that, under recommended storage conditions, CD-R, DVD-R, and DVD+R discs should have a life expectancy of 100 to 200 years or more..." HOWEVER, they do stress that more tests need to be performed. If you use the highest grade DVDs, expecting a data lifespan of 50 years or more is not unreasonable. (More on this issue later.)
See:
http://www.itl.nist.gov/div895/carefordisc/
As far as costs, the DL (Double Layer) 8.5GB disks sell for around $0.65 to $1.20 per GB, but the price is expected to drop over time. The smaller 4.7GB Disks sell for around $0.10 to $0.50 per GB.
To understand how much data is generated per hour and how many disks it will take to back it up, see the chart below. In this chart, I provide the //Number of 4.7 GB DVDs required // followed by the Number of 8.5 GB DVDs required.//
DVCPRO HD (at maximum data rate) = 60 GB pre Hour //15//8//Disks
DVCPRO50 or 720@30p = 30 GB per Hour //8//4//Disks
720@24p = 24 GB per Hour //6//3//Disks
DV = 15 GB per Hour //4//2//Disks
As you can see from the table above, at the maximum data rate, you need lots of disks, but one or two steps down and the number of disks required is not too bad. Because of the long time to burn DVDs, it would be wise to have a second computer to burn the disks. Just set up the first burn and come back some time later and change disks. It is somewhat pointless to stare at a machine waiting to change disks.
While the DL Disks cost more per GB right now, they may be the better choice because they require less disk changes. Once the price per GB of the DL Disks drops, these will be the disks of choice.
Some cautions about using DVDs for archival work:
FIRST, always use the highest grade DVDs. The really cheap stuff has been known to not last and some cheap disks delaminate after about 1 year. The link below will help you with what are the highest grade DVDs:
http://www.digitalfaq.com/media/dvdmedia.htm
SECOND, use the write once read many times DVDs (DVD-R, DVD+R). The DVD-RW and DVD-RAM will not hold the data as long as the first kind.
THIRD, do NOT use a solvent-based marker to write on the DVDs. The solvents may slowly work their way through the media and destroy the data on the disk. If you must mark the disk, use a NON-solvent-based marker and write only on the clear part in the center of the disk. Additional details can be found at:
http://www.itl.nist.gov/div895/carefordisc/disccare.html
THE FUTURE:
I can safely say that the price per GB of storage will drop every year. Also, new technologies will open new doors and generate new options. Blu-Ray single layer disks will hold 25 GB and the double layer disks will hold 50 GB. The Holographic Blue Laser disks will hold even more data (300GB) per disk. InPhase says that their disks will have a 50 year archival life and a media cost of $0.25 per GB.
http://www.inphase-technologies.com/index.html
http://www.enterprisestorageforum.com/technology/news/article.php/3523316
Even if 99% of the R&D for future storage is a failure, there are so many companies doing so much research into improved technologies, enough will come up with something even better than what I've listed so far.
SO WHAT SHOULD I USE FOR NOW?
The good news is we have many choices to select from. The bad news is deciding which is best for your needs. The correct answer depends on how much data you plan on generating per week, will the data need to survive past 10 or 20 years, how much are you willing to spend on the initial investment, and how much per GB will you be paying.
Another issue is technical obsolescence. Consider a tape backup system, you purchase a unit that works for several years. One day the drive fails and you discover that the company that made the tape drive is out of business or quit making the product; how will you recover the data on the tapes? You may find a used drive for sale OR maybe when you first purchased the drive, you purchased several of the same tape drives to be used on other systems. Another approach would be to put your second copy on a different media. Even if "xyz's" tape drive is no longer made, a second copy exists in another form.
To go into detail would take up too much space, but let me summarize some general guidelines:
TAPE: The price for the tape drive system varies and you get what you pay for. In general, the more expensive drives offer faster transfer rates, higher capacity, and most important of all, a more rugged drive. This can be important when you have massive amounts of data to backup every week. If the manufacturer publishes a MTBF (Mean Time Before Failure), this will give you some idea how long you can run the drive before you have a 50%/50% chance of a failure. Remember the real cost of this system is not just the drive, but you have to consider the cost of the media and how long it will take to do a transfer. Tape can be used in high and medium volume operations.
REMOVABLE HARD DRIVE: If it wasn't for the issue of killing a drive when you drop it, the removable hard drive would be perfect for medium and small operations. It can still be ideal if you have a second copy on another drive. (If you think about it, how often do you drop a video camera.) The random access nature of the drive makes it tempting even for large operations. Given how prices drop to 60% of last years price every year, this is a tempting option.
OPTICAL MEDIA: In a small operation with limited data every week, even 4.7GB and 8.5GB DVDs can be very practical. You will need a second computer to burn the disks and have to change the disks ever so often. After Blu-Ray Disks become practical and cost effective, this will make going optical for storage more attractive for everyone. If you record something that's VERY important and you want it to last, optical media is the best bet.
Last of all I leave you with one more link to look at. The article was written in 2004, but it is still worth reading:
http://asia.cnet.com/reviews/hardware/storage/0,39034915,39091141-1,00.htm
Bob Diaz
Wow that was a LONG post!!!
:beer: