Realistic RAID setup for shooting in remote locations?

Tom Lowe

Tom Lowe

Well-known member
If we wanted to shoot RAW 4K uncompressed on location, so we can overcrank to 60fps, what realistically are we talking about in terms of hardware, power requirements, manpower, etc?

I have seen mentioned that you will need a 10-drive RAID system to record the (350MBps-875MBps??) data coming out of the camera. How expensive -- ballpark -- will it be to buy or rent a RAID array of this type? How much power does something like that draw? How much data (hours of overcranked footage) might it store? Will a putt-putt gennie provide enough power?

What type of cable runs from the RED to the RAID array? Fiber? Is that expensive? Can it run over long distances, like hundreds of feet, so you could have your array in a motorhome, for example, with a cable running out to the camera operating on set? Is the cable sensitive to being stepped on, etc?

And finally, if you shoot 4K RAW uncompressed to the RAID array, will you be able to hook it up to a PC at night and process/compress the RAW data to 4K REDCODE RAW with your PC, so you can dump the RAID array and create new space for more RAW uncompressed?

thanks
 
I haven't seen many details about the RAID but a lot of the specs depend on if it will use 2.5" laptop SATA drives or regular 3.5" SATA drives.

Check out the Ciprico MediaVault 4400 which uses 40 2.5" 100GB laptop SATA drives but can do over 1200 MB/s, is shock-resistant, and I'm sure is quite expensive.

If the Red finds a way to make the entire drive inclosure shock-resistant, then maybe they could use 3.5" SATA drives instead of the 2.5" drives. The power requirements may be higher, but at least you wouldn't need as many drives, it would be cheaper, and much higher capacity.
 
Great questions, Tom. I look forward to the answers as well.

And, I agree it would be great to have the 4K uncompressed RAW, once recorded to RAID, be compressed by a computer to REDCODE RAW 4K to allow more space on the RAID. This would effectively make 4K Redcode RAW at 60p a reality for a lot of us that can't afford working in uncompressed 4K.
 
It seems to me that you need to have a computer with video capture card in the field to record with the MediaVault. The RED camera has a RAW output, which is going to be connect to the recording Raid system, and it should have a software to handle the recoding, like FireStore product.

I would be very interested to know more about the RED RAW output, and how RED has recorded the uncompressed 4k images they have been shooting.
 
Very interesting topic, and one I'd like to explore as well.

The scarey part for me is: 2 Hour feature... shooting 5 to 1 or 10 to 1... means how many terabytes of raw? And if you overcrank to 48fps what does that do to your datarate?

Once you have it, what's your workflow?
 
An external RAID like you are talking about Tom would use 3.5-Inch SataII or SAS drives - There simply would be no benefit using a 2.5-Inch drive in my opinion. They will have enough shock resistance in them to be moved around but probably not whilst you are recording with them unless very minimal/smooth movement. The SAS drives have faster spindle speeds of 10,000 or 15,000 RPM compared to SATAII's 7,200 RPM but whether this would be a necessity for the ammount of data going across is debatable - I'll try and find out at some point. Needless to say both have dedicated 3.0 Gbps bandwidth which I believe ought to be enough. I imagine a disk storage expansion enclosure would be a good starting point, you could begin with a few 500gb SATAII drives and install more as you go. My cousin use to build and sell rack mounted units so I'll have a word with him at some point and see if I can't discover more details about how this could be encorporated into Red. I don't yet know whether the camera needs any kind of gobetween to write to the disks or whether it would just recognise them and happily write away to them or what.
 
Tom Lowe said:
Thanks, Homer. What kind of cable will run from the RED to the array? Fiber?
Click to expand...

I do not know, it is only specified RAW output.... may be somebody saw it or ask this to Jim at the LA screening.

Homersapien said:
The SAS drives have faster spindle speeds of 10,000 or 15,000 RPM compared to SATAII's 7,200 RPM
Click to expand...

The RaptorX spin at 10'000 rpm and it is SATA 2. It might be interesting to look to eSATA possibility...
 
I'm not sure Tom I'm by no means an expert, I'll have words tommorrow and see what I can find out about it. I notice Dell's servers seem to be rather expensive - the 500gb SataII drives are listed at £310 each ($550ish). I've seen these for as little as £105. I guess Dell don't offer such cheap prices to small businesses as they do to the average consumer.
 
Homersapien said:
An external RAID like you are talking about Tom would use 3.5-Inch SataII or SAS drives - There simply would be no benefit using a 2.5-Inch drive in my opinion. They will have enough shock resistance in them to be moved around but probably not whilst you are recording with them unless very minimal/smooth movement.
Click to expand...

I agree, and have already put in my vote for 3.5" user replaceable SATA drives. I want the same throughput/connection between the camera and RAID, and the RAID and PC. This way the RAID could be used for both capturing from the high-speed data port, and for an online storage solution. I need a high-performance SATA enclosure anyway, might as well purchase the RED-RAID and have the ability to capture from the high-speed data port when necessary. I don't want to have to buy two seperate SATA enclosures....
 
I brought up whether we'd be able to use a 3.5" drive as an onboard device for the same reason Jahlion; because it would be so simple to deliver the footage to the client once you were finished, but from the conversations that ensued I don't think it'd be viable. There are too many moving parts on these types of drives which is what makes 2.5" versions better for mobility. If a casing that could enhance the shock resistance and lessen ammount of movement that could effect the recording then it'd be viable, but doing so will probably put the cost into the realm of the RedDrive anyway... pointless.

I think that a RAID setup onsite that allows you to strap together between 2 and 15 500gb SataII harddrives is about the best solution. Combine with that a means to download footage from a RedDrive in a matter of seconds as well, this would allow you to shoot remotely and have a place onsite to backup to instantly. Alternatively if you want to shoot 60fps+ you drag the RAID upto where the camera is operating plug it in and shoot direct to it. Your only other major concern is a power supply that can be used nearby and not be too noisely (For external shooting). But having this available will be necessary for your other considerations such as field monitors, lighting etc so it wouldn't be much of a problem, if you are going onto someone elses set then this would be their concern, all you'd need to do is tell them how much power you will need to fuel the RAID.

If you need to shoot RAW then you will probably need more than just the 15 500GB drives anyway, but you can usually link 3 of this setups together to get 45 x 500GB which would be enough for a heck of alot of time. I believe Mutants shot on SI's camera (Not sure if this is 100% correct) used 170TB, but if I remember reading correctly they were continuously shooting between takes - why is beyond me. I remember the ammount of footage actually recorded was around 80 hours worth - crackers!

With Red RAW footage would be about 1.628TB/hr if I'm not mistaken so 15x500GB drives (7.5TB) would give you about 4 hours and 40 minutes of RAW material. But the truth is that if you can afford the system to actually edit the RAW material then worrying about a setup for a RAID won't ever be a problem to you.
 
Great post, Homer.

A couple of things to keep in mind. If you are using this RAID system to record RAW overcranked footage on location, you won't have to worry about the sound of a gennie or whatever, because it's MOS. Also, unless I am mistaken, the date rate from RAW overcranked is going to be considerably higher than whatever the listed data rate is for 24p (350MBps). So it might come out to something more like 3 or 4TBs per hour. Is that correct?
 
I would keep an eye on Stwo. They've already demonstrated systems that in production can show over 350 MB/s on set. From there it's just a matter of ramping up the size of the Dmag.
 
Homersapien said:
I brought up whether we'd be able to use a 3.5" drive as an onboard device for the same reason Jahlion; because it would be so simple to deliver the footage to the client once you were finished, but from the conversations that ensued I don't think it'd be viable. There are too many moving parts on these types of drives which is what makes 2.5" versions better for mobility. If a casing that could enhance the shock resistance and lessen ammount of movement that could effect the recording then it'd be viable, but doing so will probably put the cost into the realm of the RedDrive anyway... pointless.
Click to expand...

I don't want to use the 3.5" drives for onboard recording, only for the RED-RAID.
 
I am planning to outfit a van with plenty of storage, probably a SAS card on a tower feeding RAIDs. The Red Team is always saying that anything can change during development, but at one time they were planning an Infiniband pipe to get data off the camera. In general fiber is pretty robust and can go long distances so running it to a nearby van or RV should be viable, in fact, it is part of my plan. As part of my due diligence I will be consulting with an IT person to see just how robust the fiber pipes are and checking with vendors for cost. No operator likes a tether but for overcrank or high end VFX heavy features there will be a demand for the "refridgerator sized RAIDs".

Blair S. Paulsen
RedOne #19
 
You must log in or register to reply here.
Back
Top