Preserving Tapeless Video:

As we move from video tape to tapeless production, the issue of preserving your videos for an extended time period raises its ugly head. What we all want is a simple, inexpensive, and 100% reliable technology that will preserve our data forever. Sadly, no such technology exists. This article looks at the less than ideal solutions to safeguarding our videos and how to work within the limits of our technology.

For those who think that somehow tape is immune to these problems, you are sadly mistaken. Tape is a storage media, but it is NOT 100% bullet proof. Tape will fail over time.

FIVE KEY QUESTIONS

When deciding on saving your tapeless video the five key questions to ask are:

1. How much do you want to save?
2. How long you need to save it?
3. How much trouble will it be if it's lost forever?
4. How much are you willing to spend?
5. How much work are you willing to do?

How much do you want to save? It's tempting to save everything, but this could be a massive amount of video in just a few years. Do you really need to save every clip? Is saving the finished product enough or do you need to save the the clips used in the video.

How long you need to save it? Saving something for less than 5 years is reasonably easy, but saving something for 30+ years is a lot harder and requires more work.

How much trouble will it be if it's lost forever? What will it cost you if your system fails to save this video? A wedding videographer could try to save all the wedding videos for 20+ years, but what are the chances that a couple will come back in 20 years and ask for a second copy? Whatever profit one might make from that one sale would be lost in all the expense of storing all the other videos. On the other hand, loosing the only copy of a video deposition in a multi-million dollar lawsuit could be catastrophic.

How much are you willing to spend? AND How much work are you willing to do? These questions are answered after you answer the first 3 questions. Increased protection comes at increased cost and work. Longer storage time comes at increased cost and work. One has to ask, "How important is it to preserve this video?"

THE IMPORTANCE OF REDUNDANCY:

The old saying holds true here, "Never put all your eggs in one basket!" If your critical videos are held on a single piece of hardware and it fails, it's lost forever. Having a second copy on a different device provides a backup in the event of the first device fails. For really critical videos, a third copy adds an extra layer of protection in case the unthinkable happens.

Assuming you have a second or third copy at your office; What happens if there's a fire, flood, or major earthquake? Could something unexpected destroy all your copies? Having the second copy stored at another location could protect you from the unthinkable.

The other issue that people fail to consider is; What happens if you have chosen a device that has a major manufacturing defect that causes loss of data when it fails prematurely? Two identical devices from the same manufacture may fail at the same time. The simple solution is to chose different manufactures for each redundant device. The chances of both devices having this exact same defect are very small.

SOLVING THE HARDWARE OBSOLESCENCE PROBLEM:

At one time, 5-1/4" floppies were the standard for storage, but today finding a 5-1/4" drive to read that floppy is next to impossible. At one time, RS-232 and Centronics Parallel were the standard interface, but today most computers don't even support that.

The standard interface changes over time and today's USB and Firewire interface may be unsupported in 10 to 20 years. The good news it that there's always a transition period where one can transfer from one older media to a newer media.

The oldest files on my computer date back to 1987. While I can't say exactly how I transferred the files from the early systems to the newest system, likely they passed from one system to another over different media.

SOLVING THE SOFTWARE OBSOLESCENCE PROBLEM:

At one time WordStar was the standard file format that everyone supported. Today, you'd be hard pressed to find any support for that file format. Just like hardware, software file format standards change over time. Today's standard file format for video may become obsolete in the future.

Like hardware, there will be a transition period to convert the files over to the new format. However, if you have to do this one file at a time, this could be too labor intensive. Clearly you won't be the only person who needs to do this, so the likelihood of a batch conversion program existing is very high. For the most critical videos, you may want to save the end result in several different file formats.

PROS AND CONS OF THE DIFFERENT STORAGE MEDIA:

It's best to think of the current storage media as a temporary holding area for your data until something better comes along. While less than perfect, today's storage media can maintain your files until something else comes along.

Hard Drives: External USB and Firewire drives offer a simple, low cost, and large area for storage. They can be stored on the shelf and plugged into a computer when needed. However, one slip of the fingers and the drive falls on the floor; resulting in the total loss of data. The importance of redundancy is very critical with hard drives. Even if you never drop a drive, hard drives have been known to crash on their own and loose data.

The bearings of hard drives have been known to freeze up when you place then on the shelf for several years unused. Thus, one should run a yearly disk scan just to make sure that the data is still intact. At the first sign of trouble from any drive, copy the second drive with the backup copy of your data before it fails. The lifespan of a hard drive will vary, but it's reasonable to expect around 3 to 7 years from a drive with normal service and even longer with partial service.

Optical Media: When CR-R disks first came out the promised lifespan was 50 to 100 years. However, some people encountered problems with disk de-lamation these reports caused people to worry that the optical media wouldn't be a good archival media. However this was a case of the cheaper quality disks making the good quality disks look bad. My oldest CD-R was burn in 1997 and is still readable.

Name brand disks, like TDK, Maxell, Verbatim, Panasonic, and Sony are likely to be good quality. Details of how to determine the source of the disk is found at the links below. In general, the deeper the blue/purple color of the disk, the better the quality.

http://www.digitalfaq.com/reviews/dvd-media.htm
http://www.cdr-zone.com/articles/recordable_dvd_quality_page_1.html

When archiving video to disks, always archive the same file to 2 different disk brands. In the event that you accidentally selected a substandard disk, the other disk should survive.

SL (Single Layer) 4.7GB DVDs are relatively inexpensive per GB, however the smaller size of the disk, means a lot of work burning to lots of disks. the DL (Double Layer) 8.5 GB DVDs will require less disks to store your files, but cost considerably more per GB. Both the SL 25GB and DL 50GB Blu-ray recordable disks look promising for the future, but the cost will need to come down for most of us to consider this option.

A life span of 50 to 100 years for optical disks may be possible, but one must protect the disks from excessive light, heat, and cold. Studies show that both direct and indirect sunlight shortens the life of the data on the disks. So store the disks in a dark place.

Flash Memory: As flash memory prices continue to drop, many videographers who use cameras like the Panasonic HMC-150 simply save their original video in the SDHC card it was first recorded on. Even if you don't use SDHC cards, USB Flash Memory sticks can be used for storage.

Unfortunately, very little data exists as to the archival quality of flash media. Assuming good quality flash memory, the storage life may be 10+ years to 40+ years. I just wish I had more information to be able to talk about which flash memory chips will survive that long and which chips won't.

Of interest is the SanDisk WORM (Write Once Read Many) memory. This may prove to be a possible solution for the future. Sandisk reports that the data is expected to survive up to 100 years

Tape Backup: There seems to be a sense of irony for me to suggest that one could go tapeless by backing up to tape. However tape back up systems are a possible means of storing your long term data. Done properly, tape is expected to last about 30 years. Several things are very important to do when using tape.

First, one must have a minimum of 2 working tape drive units. In the event of 10 or 20 years down the road, if one drive fails, you will need the other drive to read your data. It's possible that in 10 to 20 years the standard for tapes may change and you will not be able to purchase a tape drive that can read your old tapes.

Second, when writing to tape ALWAYS verify after a writing. There are several cases of people who ran backup tapes from the server and did not verify after a write. The tape drive went through all the motions, but due to a problem, wrote nothing to the tape.

Third, tape has a limited number of passes it can be written and read from. This is not going to be an issue if you write to a fresh tape and check it every few years.

FINAL COMMENTS

Consider using 2 different storage medias for long term storage. If copy number 1 is on a USB Hard drive and copy number 2 is on an optical disk, your odds increase for being able to get at the data years down the road. It's impossible to predict the future, but by having 2 different formats, there are 2 different ways to read the file in the future.

There's always a possibility for a virus attack. Even if you are attacked, any data offline and in storage is safe. However, one should run a good virus check program before you plug in any offline storage. Maybe it's best to have the computer that deals with archival data a "virgin". That is, it never connects to the internet, networks with other computers, or runs any other program except the software used to archive and check the data.

With care you should be able to preserve your video files for a long long time. The important thing to remember is never be careless with your data; better safe than sorry.