documentary storage solution

C

Corby Evenson

Member
I was hoping someone could help me out in finding a proper storage solution for a project I am about to start work on. I’ve done smaller projects but this is an hour long documentary working with dvcpro hd 720 24pn, shot on an HVX200 to P2 cards, so my limited storage won’t cut it.

I’ve been reading about raid and trying to soak up what I can from message boards but I thought I should post and see what the community has to say to me specifically.

I have a mac pro so part of me thinks that i might be able to get away with internal drives instead of an external raid set up. I am concerned with speed and redundancy though. So then I think Raid would be the way to go. I don't know what fits the project. I'm not sure if I should get a raid 1 or raid 0 with some sort of separate backup drive I copy to every day. I've been looking at the G-Raid3 and the Caldigit VR online, but I don't think I know enough to be able to make an intelligent decision. I do like the fact that the Caldigit has removable drives so that when this project is done I can simply swap the drives. Unless that is the wrong way to archive a large project like this. But really, I'm not even sure if the GRaid or the Caldigit options are the best for me.

If I’ve missed something or I am mistaken, please forgive my ignorance as I’m still trying to get a handle on things.

There will be over 100 hours of dvcpro hd 720 with a bit of hdv thrown in.

I’m wondering what some of you are working with and what you like. If you were in my situation what would you suggest?

Any help would be greatly appreciated. Thanks.
 
Well, first off, I'd ask what your budget is. Second, is this something you need to do often? Third, how many HDDs in your MP now? If you go with your internal drives, you can use Mac OS X's Disc Utility to put them in a RAID 1 (at least 2 HDDs, mirrored = 1 HDD is back-up only), RAID 0 (at least 2 HDDs, striped = data is written across all drives in the set up), or set it up as a RAID 1+0 (IMO, at least 4 hdds to make any sense, 2 RAID 1 sets striped together, effectively using less than ~half total capacity). I don't remember, but I think if you RAID your internal HDDs using DU, you have to format them, even when you change the setup/replace HDDs.

Another option is to get a HW-based RAID controller card (CalDigit, HighPoint, etc). Some of these will give you the option of connecting to internal+external drives, have a mem cache, act as a Port Multiplier (meaning many, usually 4, drives connected over one port), are bootable, etc. The lower-end ones I've seen (Highpoint) have a browser-based UI that you need to go through in order to set-up your volumes, mount/unmount, etc. Since running a RAID can strain the CPU (especially on older machines), these cards take care of the processing instead. The higher-end cards offer more features (eg. support RAID 0, 1, 3, 5, 6, 10, 50, 60)... usually include their own SW UI, have better tech support (so I've heard), etc. RAID 5 (or 50) offers, I think, what most people want out of a RAID setup - capacity, performance, redundancy. You need at least 3 HDDs for that.

As for the CalDigit card, I think it can only be used with the CalDigit HDElement, maybe some other CalDigit enclosures, but not other enclosures. It sounds really good, though, I think even some guys over at REDUSER, and probably here, use it.

FWIW, my MP is set up like this:

Using DU:

1x 500GB boot drive (int)
1x 500GB media drive (int)
2x 500GB RAID 1 (int) for back-up

4x 1TB (4TB total) RAID 0 (ext) scratch disk connected by eSATA

Now, I know this isn't the best or safest or fastest set up, but it kinda just evolved like that. I'm just an enthusiast, so I certainly don't put my drives through half as much as some of the other guys around here do, but so far it's been great. *knocks on wood* It's somewhat of a pain to have to manually back data up, but like I said, I'm not a pro, so I can afford to sit sround doing that.

If I had the cash, I would definitely replace my HBA with a HW-based RAID controller, add another enclosure, replace my boot drive with a raptor, get a RedRocket, a 4k display, buy a house, a car, a... :)

Last weekend, my brother gave me about 5hrs of H.264 footy of some doc he was doing for fun with his friends (he wanted it 1hr long, entirely cut, CC'd, and delivered, w/ custom layered menus in DVDSP no less, in THREE days... I did it, but honestly, there are a million things I would've liked to have tried with it!). Of course, the biggest time periods went to ingesting (USB2.0 - why?), waiting for it to transcode to PRORES (faster than realtime, ~8-10x the size), render out of Color/FCP/AE, and ouput from Compressor. So for each project clip, I have 2 back-ups. Of course I didn't transcode every minute of footy to ProRes, but I did enough to realize that I could use a bigger media drive!

So yeah, not sure if any of this helps... but I've been fine with RAID 0 for about a year and a half. As for long-term achival solutions... *SHRUG* LTOs? BluRays? DVHS? lol

EDIT: http://www.dvxuser.com/V6/showthread.php?t=174747 This can get you started.

BTW, all HDDs can fail, regardless of brand, but I find that certain enclosures work better than others.
 
Last edited:
Something to keep in mind is the difference between redundancy and back-up. RAID1 offers redundancy but it does not offer back-up. If you accidentally delete a file or a file becomes corrupt in a RAID1 environment the file is gone. If you are working and there is power surge that fries your system your files are gone. Copying all of your files onto another set of drives and storing those drives at another geographical location (ideally) offers back-up. Copying your project file onto a thumb drive or a remote server on the internet offers back-up. Redundancy is good for minimizing down time in case of a HDD failure while a back-up is good for keeping an extra copy of files safe incase something catastrophic happens.

For DVCPro HD and HDV any FW400 drive will be enough to pull a few streams. FW800 will obviously be faster and eSATA faster still.


-A
 
Thanks for you input.
I was also wondering about how large of a drive I will need. Like I said it's DVCPRO HD 720pn and right now all of the P2 cards have been copied onto a 500 gig external harddrive. There will be more footage coming as well as around 10 hours of hdv. After capturing all the footage I don't want the drive to be full. Am I looking at like a 3Tb caldigit VR in RAID 1 or a 2Tb in RAID 0? Also even if I have a RAID 1 setup (or RAID5 for that matter) would backing up to another physically separate drive everyday be necessary?
Thanks
 
kinda hard to give you a space estimate if you don't have an estimate for how much footage you'll have...

yes, for digital acquisition it's a good idea to backup everything to a separate disk each night, but you only need to copy files that are new or changed in the last day.

the 2TB in RAID0 will probably be fine for you, but I'd keep that original 500gb drive with the masters on it as your backup (if it can hold it).
 
You must log in or register to reply here.
Back
Top