State of the Data, 2018: how are you managing the terabytes?

I also have a RAID 5 Edition Thunderbay, but it sounds like I've had a different experience. The SoftRaid app that comes with the enclosure is pretty handy, since it alerts me in advance if any drives in my RAID array (or, drive plugged into my computer, including flash drives) are about to fail. That's saved me a time or two, and I've been able to make emergency backups. It's also nice knowing that if one of those four spinning drives fails, I'll still be able to recover my data.

Secondarily, I typically only keep the last six months of files on my RAID. Everything else, I move onto a pair of spinning drives (each with a protective case). They are duplicates. One of them stays in a filing cabinet next to my desk, and the other goes across town to my friend's closet.

And lastly, I've heard good things about a cloud-based service called BackBlaze (might not be helpful with the OP's DSL situation). But I'm curious if anyone else has used it and can share their experience: https://www.backblaze.com/
 
Back in 2000 I used DVDs and also kept the tapes.
Now I use external drives and keep a copy on two different drives and a list telling me what is where so I can find it. And from time to time I also copy files to an external drive at a different location giving me one more copy, in case the first location burns down, gets robbed etc.
About every 2-3 years I replace my internal drives so then I also keep the ones I took out with whatever data is currently on them.
This system has worked on 4 workstations now for many years.
Data really piles up recording long multicamera events
 
I think the "industry standard" is there should be at least three copies of the data at any one time and at least one copy at a different location(off-site). When you get into insured movie production, it gets VERY stringent and safeguards have to be adhered to. We did a BTS/EPK shoot for a major theatrical feature, nothing to do with the release itself, and even that had very stringent requirements requiring a dedicated data wrangler to offload, copy and verify the data to encrypted drives that were shipped seperately and the subsequent drive(s) could only be shipped after the first drive was received and all data was verified. Kinda like how the POTUS and VP will never be on the same plane together at the same time.
 
I get nervous with drives as well, although starting out as a freelancer I use pairs of 6TB in dual docks for backup and archival. Just have a system in place to regularly spin them up and do a full sector scan to get ahead of any failures.

When I worked at small facilities I swore by LTO. I find there's economically a tipping point where tape becomes a cost effective option if you generate enough data. It has at least a 10 year shelf life when stored properly. I never had a tape fail out of a combined 300 tapes. You do have to factor in a new deck every 5 years or so I found. As per drive backups I'd recommend two dual tape catalogs for redundancy, one stored onsite, one offsite.
 
I hate optical media. I've been super careful to only get the best quality recordable CD, DVD, record at low speeds etc., and I find a very, very high failure rate. So for me, it's external hard drives. I used to do RAID1, but I've had failures there too, including the drives as well as the controller (Western Digital - stay away from that garbage). I've had individual HDD fail too, both external and internal - f.ex. my iMac (thank you, ApppleCare) had to have it's internal HDD replaced 4 times (Seagate - stay away from that garbage).

Now, I use the exact same method Battuta uses - external drives copied using SuperDuper. Here is what I find - HDD tend to fail in series, so I never buy them from the same lot. There was a spate of bad Seagate drives that had to be replaced often, a known problem (the case with my iMac), and so on. So now, I'll use a Samsung and Hitachi drives to copy to each other. I use 8TB drives (which so far according to statistics) have low failure rates. I find 8TB drives a good compromise - size and price and reliability. Just copy those suckers - what are the odds that both a Samsung and a Toshiba will fail at the same time? Low. What are the odds that two Samsungs or two Seagates or two Western Digitalas fail at the same time when they come from the same batch - HIGH.

So my practice - never buy HDD from the same batch to copy each other, odds are they'll fail. Also RAID arrays can fail due to both hardware and software (f.ex. controllers), so why play that game - and besides, RAID encourages you to use the drives that are identical as it's easier and supposedly more "reliable" what with the same power saving profiles and so on (big problem with WD drives - failures of power saving modes). Except when the same batch HDD fails, your RAID that's from the same batch also goes tits up. So the heck with RAID (again, I had WD bassed RAID1s fail more than once - and data was not recoverable).

External 8TB HDD from different manufacturers (or at least different batches) and just copy using reliable software like SuperDuper - FTW!
 
I agree with Morgan except I have two separate backups. So, working and backup 1 in the house and backup 2 offsite. Backup 1 and 2 get rotated almost daily and I use Chronosync for backing up which means I barely have to think about it.
 
You must log in or register to reply here.
Back
Top