Archiving
- Thread starter scarfhead
- Start date
Barry_Green
Moderator
For a terabyte? Uh, yeah. That's a lot of data! Eventually the holographic video disc should come out, starting at 300gb capacity and growing to 1.6 terabytes.
But to handle the kind of data quantity you're talking about, data tape's about the only currently-available viable option...
But to handle the kind of data quantity you're talking about, data tape's about the only currently-available viable option...
KnockOutFilms
Active member
Barry_Green said:
Isn't a terabyte about right for a feature. That's only 10:1 at 720p correct? This really does highlight the short term issue with archiving that everyone that wants to shoot a feature will have. I'm hoping to use a bunch of drives. DLT would be a bit expensive to ramp up I think.
Chris
smelni
Veteran
I think DVD (single or dual layer) is a very real option.
You just need to add it into your workflow. Forexample, shoot 4 GB card until full. Assistant dumps to laptop while you start a new card. After the dump burn the results to a DVD and label - Burn a second copy if you need to for redundancy. Make sure you label your disks well with a serial that corresponds to the shoot log for that disc. Put the disc in a large booklet. Done.
One terabyte would require about 250 dvds for a single copy or 500 for double copies. at one dollar a dvd that is a very very real option and frankly is way faster then low end tape solutions and allows quick access to the files instead of scrubbing through tapes.
You just need to add it into your workflow. Forexample, shoot 4 GB card until full. Assistant dumps to laptop while you start a new card. After the dump burn the results to a DVD and label - Burn a second copy if you need to for redundancy. Make sure you label your disks well with a serial that corresponds to the shoot log for that disc. Put the disc in a large booklet. Done.
One terabyte would require about 250 dvds for a single copy or 500 for double copies. at one dollar a dvd that is a very very real option and frankly is way faster then low end tape solutions and allows quick access to the files instead of scrubbing through tapes.
Flintstone
Well-known member
250GB drives are dirt cheap! Why not use those? Pretty soon, the larger capacity drives are going to follow suite. If you're looking to keep the footage handy until the project is finished, this could probably be the best. If you want to keep stock footage for later use, then as Barry said, data storage tapes such as DTL may be an option. You should weigh the price per gigabyte of tape and hard disk. You may be surprised at the figures.
smelni
Veteran
hard discs are not a reliable store - one drop and you are done. They can overheat, mechanical failure etc. i work in the IT world and we NEVER rely on drives for storage of data for more then a day (hence nightly backups).
Tapes - are slower, more expensive and do not allow random access
IMHO - DVDs are the way to go especially given the great 4GB size we will need to use with the P2 cards - fits perfectly and its very cheap.
Tapes - are slower, more expensive and do not allow random access
IMHO - DVDs are the way to go especially given the great 4GB size we will need to use with the P2 cards - fits perfectly and its very cheap.
Flintstone
Well-known member
I know what you mean smelni, but consider that DVDs are relatively small in capacity. Imagine the backup times it will use. Besides, failure rate of hard drives aren't what they used to be. MTBF are very high, and how many times does one drop a hard drive? I also worked in IT (17 years), and sometimes, just sometimes, far fetched ideas are probably the best solution.
However, I would not advocate this method for broadcasters, and medium to large production houses where many hands contribute to potential disasters. But for the independent producer, where manipulation is restricted to the handful, using low-cost reliable brand hard drives might just be the fastest and most economical way to go.
I personally have been doing this for years. I have multiple hard drives in removable drawers that hold up various projects. When one project is on standby, I just remove the disk, store it in a safe place, and use another hard drive for the next project. It's cheap, practical, and efficient.
However, I would not advocate this method for broadcasters, and medium to large production houses where many hands contribute to potential disasters. But for the independent producer, where manipulation is restricted to the handful, using low-cost reliable brand hard drives might just be the fastest and most economical way to go.
I personally have been doing this for years. I have multiple hard drives in removable drawers that hold up various projects. When one project is on standby, I just remove the disk, store it in a safe place, and use another hard drive for the next project. It's cheap, practical, and efficient.
smelni
Veteran
Flintstone - i agree with a lot of what you are saying. Its good for smaller productions which dont have environment issues (no place to safely place drives).
I would put the footage on 2 drives though to have a redundancy. Drives dont fail often but they DO fail - and it only takes once to ruin your life (for a short time).
No one should assume a drive failure will not happen to them - if you use them often enough it WILL happen - guaranteed.
Also, burning a dvd is nearly as fast as copying 4 GBs to a hard drive with a fast DVD drive. if you work this into your on set process there shouldnt be an issue. hand the card off, have it burnt by an assistant
I would put the footage on 2 drives though to have a redundancy. Drives dont fail often but they DO fail - and it only takes once to ruin your life (for a short time).
No one should assume a drive failure will not happen to them - if you use them often enough it WILL happen - guaranteed.
Also, burning a dvd is nearly as fast as copying 4 GBs to a hard drive with a fast DVD drive. if you work this into your on set process there shouldnt be an issue. hand the card off, have it burnt by an assistant
Flintstone
Well-known member
True!smelni said:
Ah! Being able to delegate! :thumbsup:smelni said:
Optical media are not very good for archival. The dye deteriorates after a few years. It takes too much time to restore, check validity or copy. Many recording engineers have lost material due to this reason. DVDs are especially problematic due to high density, not to mention mistakes in the archival process with so many disks.
I would keep both tape and hard disk backups in different places and create new backups every 5 years or so from both sources for checking and to preserve compatibility with new tape/hdd formats. The backup process will be very fast with these.
If the content was extremely important I would also keep it online (accessible by a pc) and checked automatically in regular intervals.
You never know
I would keep both tape and hard disk backups in different places and create new backups every 5 years or so from both sources for checking and to preserve compatibility with new tape/hdd formats. The backup process will be very fast with these.
If the content was extremely important I would also keep it online (accessible by a pc) and checked automatically in regular intervals.
You never know
smelni
Veteran
Xenophon - I am not sure what your point is - you say that dvds deterioriate after a few years but then you are recommending a process that requires a check every 5 years.
Seems to me , you can accomplish what you want for less money with dvds - you can also get long life discs if you think your footage will be needed in 80 years when we are watching super duper duper HD
Seems to me , you can accomplish what you want for less money with dvds - you can also get long life discs if you think your footage will be needed in 80 years when we are watching super duper duper HD
My point is that you can depend on tape and if you need fast recovery and another cheap security measure you can use normal HDDs and backup both every few years since it so cheap and fast.
You cannot depend on optical media. Especially not on 250 DVD disks burned by a human. How perfect will this process be? What about large files? Volumes?
If burning 250 dvs is cheap and takes so many hours, your time should be worth nothing. It is an insane ammount of time. And you will do those at the fastest possible speed I bet. Another way to get an inferior burn and compromise the procedure. A hard disk or tape backup ia a single step and quite fast procedure. How about restore time? How can you check validity of the DVD backup in the future? Optical media is the worst way to backup large ammounts of data, period. I haven't seen a tape of an archived HDD failure in my life. I haven't even seen a hard disk fail recently. I have seen 100s of CD/DVD backups failures. Studies on the subject of cd archival proved that in a large percentage they couldn't tell what went wrong! The disk surface was ok, geometry was ok, original verify worked, and it still stopped working after 2 or 3 years. Moisture, light, pressure, organic dyes, sensitivity in scratches and weak structural dynamics, everything is against the safety of a backup. If you backup on DVDs you are playing with fire.
You cannot depend on optical media. Especially not on 250 DVD disks burned by a human. How perfect will this process be? What about large files? Volumes?
If burning 250 dvs is cheap and takes so many hours, your time should be worth nothing. It is an insane ammount of time. And you will do those at the fastest possible speed I bet. Another way to get an inferior burn and compromise the procedure. A hard disk or tape backup ia a single step and quite fast procedure. How about restore time? How can you check validity of the DVD backup in the future? Optical media is the worst way to backup large ammounts of data, period. I haven't seen a tape of an archived HDD failure in my life. I haven't even seen a hard disk fail recently. I have seen 100s of CD/DVD backups failures. Studies on the subject of cd archival proved that in a large percentage they couldn't tell what went wrong! The disk surface was ok, geometry was ok, original verify worked, and it still stopped working after 2 or 3 years. Moisture, light, pressure, organic dyes, sensitivity in scratches and weak structural dynamics, everything is against the safety of a backup. If you backup on DVDs you are playing with fire.
smelni
Veteran
Xenophon - all very good points - i agree.
I think the question is are we talking long or short term. And what is the budget to do this. If you are talking short term - during the production and into post - then dvds are a good low cost solution.
If you are talking long term archival - i agree - tape is better - albeit more expensive
I think the question is are we talking long or short term. And what is the budget to do this. If you are talking short term - during the production and into post - then dvds are a good low cost solution.
If you are talking long term archival - i agree - tape is better - albeit more expensive
Ernest_Acosta
Well-known member
I think right now being that the storage technology (blue ray, HD DVDs) hasn't been released to accommodate the HVX, cheap hard drives is the best way to go. Four 250GB drives gives you a terrabyte, now backing that up is the problem. Panasonic should address this problem by allowing folks who purchase an HVX to store HD footage at a storage house for a low cost. That way we can keep our hard drives but know that we have a backup at their facility to access when needed. I'm sure there are companies out there that you can store HD footage but my guess is that it is not cheap. With Panasonic offering low cost backup storage for people who purchase an HVX, they will enhance their product by assisting the filmmaker with backup issues. After all, it is their product that we have to solve a storage problem that wasn't addressed by them. Sorry Jan, you did address it in your presentation but that technology is not here now.