DVCPROHD Archiving

M

mgreenberg01

Member
I'm starting to shoot a lot of HD Content on my HVX200. My P2 Cards are getting maxed out. I have access to a AJ-HD1400 deck. Does anybody have experience transfering DVCPRO HD Content from the 200 camcorder to the new 1400 deck. Do I have to load the content into my Velocity HD Editor Firewire and then output it to the deck via Firewire or SDI. Man this HD stuff is awesome.

Thanks,
Matt:nads:
 
At the price of that deck I would bet there are not a lot of people with experience (certainly not me) . . . Nice that you have access to one though.

Maybe a more practical solution might be archive to external HD, DVD/Blu-ray, or tape (as in LTO-2 or LTO-3). Drives are probably cheapest in the long haul--LTO tape is pretty fast (I've been going the tape route since April '06).

Best.
 
Yeah, I went to Best Buy and got a couple Western Digital MyBooks (1 TB each) for $225 each. I don't like Best Buy but they had a better price than even Newegg. Depending on how much you shoot, a TB or 2 should last quite a while.
 
i just keep a steady supply of large firewire800 drives and just dump the footage then shelve the drive after the project is done. drives (at least in my experience) are just as reliable in the long term as any other medium. and at like $200 per drive. its way less costly in time than any of the tape or blue ray options out there.
 
agreed, and a 500gb sata internal is around $100, and an extra $20 or so for the adapter. If you plan to shelf the drive anyway, I'd go internal with thet adapter.
 
I'm going to say this again, as I always do in these archiving threads.

Hard drives are NOT a reliable long term archiving solution.

Its not a matter of IF you will get burned, but WHEN.

Blueray, LTO-3 are much better in this regard.

If you are only storing a project for a few months, then fine, but if you are using a hard drive, then you are rolling the dice.
 
sounds like good "once-bitten, twice-shy" logic there matt. but ive had drives on shelves for over 10 years and not one (by a reputable manufacturer) has ever gone belly up on me. but ive had cd's, dvd's and various formats of tape all go bad.

i guess it all comes down to personal experience...
 
or professional experience.

What do banks use? Department of Defense, major financial industries that are faced with the need for reliable archives?

Tape.
 
mcgeedigital said:
I'm going to say this again, as I always do in these archiving threads.

Hard drives are NOT a reliable long term archiving solution.
Click to expand...

What factors can destroy data on HDD when you just use it for store in a shelf for archiving - let's say a clean place with not too much changing temperature and humidity?
 
What do banks use? Department of Defense, major financial industries that are faced with the need for reliable archives?
Click to expand...
i dunno. i wonder if tape may be their chosen method simply because it always has been. its what they know, what they trust and what fits into their existing workflow/culture.

and we need to also take into consideration that those organizations are storing different kinds of data than we do in this industry. recovering a text file or a photo is one thing. recovering a 120gig video file is another. as is restoring a full 700gig project file - in place, as it was last left.

ive used tape in the past and for my workflow it simply didnt make sense. GRANTED, that was years ago and im sure that the newer tape process is a great deal more nimble

and again. i just have to trust my personal history. over a decade and probably 50 drives at mine or my clients locations. not a single failure.
 
Last edited:
Honestly, you just need to do what works best for you. Personally, I like the hard drive option, but your preference is what you're going to do anyways.

What I like about something like the Western Digital MyBook is that you can plug it in when you want and if it's full, you can get another one. Store the full one on the shelf. The likeleyhood of that drive not firing up when you go to use it in the future is very unlikely, especially if you don't have it constantly running. I have hard drives from 15-20 years ago that I am still able to retrieve the original data. Hard drives are very viable options for archiving. You use one every day to edit and use the internet, what's so different about drives for storage?

But to each man his own. Do what works for you.
 
Whenever I buy a drive, I buy two of the same size, so I can always backup.

However, when Bluray becomes affrodable, I'm going to go with that for archiving.
It's good that you can't change your archive.

Either way you go, there are two important points to remember:

1. Keep one backup in another location.

2. After you make your backup, test it to make sure it really is a backup.

Les
 
mgreenberg01 said:
I'm starting to shoot a lot of HD Content on my HVX200. My P2 Cards are getting maxed out. I have access to a AJ-HD1400 deck. Does anybody have experience transfering DVCPRO HD Content from the 200 camcorder to the new 1400 deck. Do I have to load the content into my Velocity HD Editor Firewire and then output it to the deck via Firewire or SDI. Man this HD stuff is awesome.

Thanks,
Matt:nads:
Click to expand...

Matt:

To guide the thread back to your original question, yes, I have extensive experience backing up HVX-200 footage to the Panasonic AJ-HD1400 and 1200. The production company I was working with had one of each. It got the point where when we were shooting sitdown interviews at their studio and eventually on some locations with the deck, we would run the FireWire output of the HVX directly to the deck and just shoot tape.

As far as I know, we only recorded live signal from the HVX to the deck but I guess technically you could use the HVX to playback material to the deck as well off of P2 cards.

This is a viable workflow if you don't mind renting decks or own one, which we did. For my own material, I am storing it on my RAID and a back up drive, but I agree, this is not a viable long term, reliable solution. I have drives go belly up on me at least once or twice a year. I am told that it can even be that the lubricant in the platter bearings can cake up, causing failure. Hard drives are about as hardy as rice paper IMHO and everyone using them as their only method of backup is fooling themselves if they consider this a viable long term storage method.

I agree with the some of the other posters , what do the studios, big business and the government use for long term storage, tape and film. Even microfiche is a viable long term storage for documents and film for moving images. Tape, I am not so sure will last for more than 10-15 years, by this, I mean DVCProHD tape, not LTO. Nobody knows yet, the format is too new and will soon become outdated anyway.

I think that much of the material created since the digital age began will eventually be lost forever. Most storage experts agree. The only viable digital backup plan that isn't tape involves cloning digital media to multiple copies on a regular yearly or sooner basis. Hard drives are fragile. Solid state memory is better with no moving parts but once again, nobody knows how long they will remain as viable storage mediums because they are too new to tell.

Best,

Dan
 
As an editor, I will be biased towards archiving source footage on hard drives. I then save a master of the edited project on a drive as well as a tape. From what I've heard and seen with consumer dvd's and bluray is that the media is only good for a certain period before the dyes they are made from start to degrade. I'm not sure this is true with bluray, but I wouldn't want to take a chance on it. Hard drives that are simply used to archive and spend their whole lives on a shelve should be fine, but its really personal preference. Sure, if a drive constantly has power going to it, its surely to fail at some point as the parts are always moving and hot. I stick to my method of archiving source footage to hard drives and masters to both hard drive and tape.
 
mcgeedigital said:
I'm going to say this again, as I always do in these archiving threads.

Hard drives are NOT a reliable long term archiving solution.

Its not a matter of IF you will get burned, but WHEN.

Blueray, LTO-3 are much better in this regard.

If you are only storing a project for a few months, then fine, but if you are using a hard drive, then you are rolling the dice.
Click to expand...

If you use two drives for everything it's perfectly fine. 2x bare drives is still the cheapest way to go.
 
jdert said:
From what I've heard and seen with consumer dvd's and bluray is that the media is only good for a certain period before the dyes they are made from start to degrade. I'm not sure this is true with bluray, but I wouldn't want to take a chance on it.
Click to expand...

I have had approximately 30 commercial and self recorded CD/DVDs go bad from disc rot in the past few years. Some DVDs that a grapics company burned for me of my company stationary are only 4 years old and are already turning orange from the disc rot.

Blue Ray will be just as susceptible to disc rot unless Sony has come up with a better substrate sealing routine and the material is not susceptible to oxidation. Ask any data recovery expert or company how reliable hard drives are and you will get laugh, hard drives are notoriously fragile and unreliable. As any repair specialist will confirm, it's not a matter of IF your hard drives will crash and burn, its a matter of WHEN they will crash and burn. Not using hard drives over the long term can be just as dangerous as leaving them running. MTBF numbers are just a running guess really.

The only semi permanent solution is a regular schedule of digital cloning to new media formats as they become available. I would be willing to be that many of you that own hundreds or thousands of CDs or DVDs, if you went through all of them today, you would find at least one or two with disc rot/oxidation and some of you might be surprised to find many that have it. We were all fooled into complacency when CDs and digital media were introduced that the media would "last forever" Ha, what a joke. The only solution I see are the 14k gold DVDs/CDs that places like Mobile Fidelity used to issue. Although the metal disc can't oxidize if its gold, the plastic that holds it can still degrade.

Best,

Dan
 
Hard drives have a failure rate of around 2% per year. I don't know that I'd call that "notoriously fragile and unreliable"; I mean, if you're talking about dribbling them like basketballs then yes, but as far as archiving and sticking 'em on a shelf, I think they're probably a fine alternative. They just can't be beat for cost, efficiency, ease of use, space savings, etc; the only real knock is whether you consider them reliable enough.

Google did an exhaustive study of hard disk reliability and their conclusions are basically that "Disk drives are generally very reliable but they are also very complex components. This combination means that although they fail rarely, when they do fail, the possible causes of failure can be numerous."

The key to hard disks is to recognize that once you get an error, abandon ship -- a drive that's giving you errors is likely to only get worse, so if you get an error, get out! Other interesting conclusions are that they found very little correlation between failure rates and either high temperatures or higher activity levels. They also found that certain models of drives were far more prone to failure than other models. They didn't release those results however, choosing to keep that information proprietary.

This was a study of over 100,000 consumer-grade hard disk drives, both SATA and PATA, 5400 to 7200RPM, from 80GB to 400GB. They found that over the course of a year, the average success rate was around 99%. But from 2 to 5 years of service, the failure rate was around 6% to 8%. That's not absolutely bulletproof, of course, but it's probably at least as good as data tape.

And if you eliminate the low-reliability drives from the mix, the failure rate should plummet (meaning, good drives should be even far more reliable than the above numbers indicate). And if you go with data redundancy, the failure rate should practically disappear.

I'm not telling anyone TO archive on hard disk; I'm only saying that I think that people sometimes fear it for reasons that may not be statistically valid. If you think a 2% failure rate over a year, or 8% over 5 years, is way too high a risk, then you are now prepared to make your decision, armed with a substantial data study. If you think hard drives are so inexpensive and easily redundant that you are willing to consider using them to archive, I think a strategy of employing duplicate drives should reduce your NET effective storage risk to less than 1% over the course of five years -- and considering the incredible ease, cost benefits, speed, and instantaneous data access that hard disk archiving gives you, it certainly should be at least given due consideration.

For interim archiving, I'm comfortable with hard disks (and that doesn't mean that you should be; everyone has to establish their own tolerance for risk). For deep archiving, I think a combination of hard disk + data tape provides the best of all worlds. I'm not sold at all on today's optical disc for archiving, and I think archiving to DVCPRO-HD tape is the worst way to go -- unless you have a client who demands delivery on DVCPRO-HD tape. Otherwise, forget that -- data tape is much cheaper, much faster, and stores more information (metadata, clip-based file structure, all the "goodness" that comes from working on a file-based acquisition system). Optical discs are too expensive, too small, and too slow to be practical.
 
Hi Barry:

Wow, I am statistically extremely unlucky then. So far this year, two client FireWire drives bit the dust (both different brands, bought from different retailers), my PB 1.67Ghz system drive ate it and my wifes iBook drive bit it.

We baby our drives and computers, never subjecting them to shock, heat or anything bad, they just sit in our nice offices. We shut them off when not using them and pack the FW drives up and keep them in a cool locked closet. Last year I had two drives die that were Fox projects right before delivery date. We had a backup of the sequence but lost all of the media and had to restore from a second copy, what a pain.

Optical media in my experience, is exhibiting some of the same characteristics as Nitrate film stock, the problem isn't bad yet, but eventually we will be in a catastrophe. DVDs and CDs do not consistently last very long. When CDs were introduced in the mid 80's, I was in the audio business and I can fondly recall how everyone was touting that CDs would last basically "forever" without abuse. Pretty naive thinking.

The DVCProHD backup thing was only done because the company had the decks laying around, they were only used for ingest so they weren't always working on the on-line bays so we put them to use. We had a Quantum tape drive as well and did not like the experience of using it and trying to retrieve files. It was slow, clunky and not that great, although long term, it probably is reliable.

Best,

Dan
 
There is no 100% option. Unless you are like Disney that has 2 copies of all films and data stored in 2 different locations. But even then...... :)

FWIW, I have recently added a disclaimer on our job proposals stating "we cannot be held liable for lost data due to circumstances beyond our control".
I've also explained to our clients that I will keep the backups at our studio for their convinience, but in case of fire or theft, we're off the hook. They all understood and continue to leave the drives with us.

I know this isn't an iron clad clause, but the main benefit is this educates the client that we are recording DATA and not just video. This also gives me more clout when suggesting that the client pay for a backup drive for their data.
I suggest you guys start implementing something like this if you haven't already done so.

Yeh, I'm pretty particular about backing up important data, but shlt happens. There is no perfect option. Even backing up your backup's backup has a downside, it's called therapy! :)
 
You must log in or register to reply here.
Back
Top