Securing your RED-RUSHES

Nick_Shaw

Nick_Shaw

Well-known member
Here's something I've been thinking about, and thought I'd put out there to see what other people think. One thing which is going to concern a lot of people when using a Red camera is the security of their valuable rushes. People have been using film and tape for a long time, and feel relatively confident with the tangible nature of those media. Data on a hard drive is more of an unknown quantity for a lot of people.

Assuming you are using RED-DRIVEs to shoot onto, you are going to want to copy the data off those, erase them, and then re-use them. It's that word 'erase' which is scary! You need to be very confident that your rushes are safely stored on another drive before you hit 'OK' to clear your RED-DRIVE.

I know this could be accomplished with a laptop on set, and a pair of Firewire drives (second one for redundant backup). But that requires somebody to operate it, and take the responsibility of saying "yes I'm happy to erase that RED-DRIVE for re-use".

I'm thinking of some kind of 'black box' downloading unit. This is not necessarily something Red need make themselves, but maybe something third party developers could consider. I envisage something which looks rather like a battery charger, with multiple docking bays for RED-DRIVEs, each with a set of LED status lights. The unit would have two removable drive modules which could be sent to the edit suite at the end of the shoot. It would download the media from the RED-DRIVE to both drives, then verify the files on its drives against each-other and the originals on the RED-DRIVE. Only then would it erase the RED-DRIVE and indicate with the status LEDs that that particular module was ready to be removed and re-used.

Other additional possibilities might include a built-in tape drive (DLT, LTO, AIT or whatever) which could automatically archive the media as it went along, and maybe even a basic version of REDCINE which would automatically make "one-light" HD/SD copies of the media in a selected format.

What do people think?
 
I know where you're coming from, but I think it's a little too soon to speculate about the workflow and backup of footage until we know exactly what kind of interfaces the RED-DRIVE will provide.

I'd love to make a RED Genie, that's for sure! :)
 
yes nick - this is the problem of every digital tapeless production. i am still wondering, that many people don´t know all the problems and affairs with a digital cine production. it is the same with Panasonics P2 and many other formats.
 
currently - for film scans and film telecines to hard drives ..
the loaner drive come in - i transfer to a external 1 terrybyte raid 0 .. i then transfer again to a single hard drive ( back up ) = the single drive goes to offsite media storage company ...
i have to say i do not check every second of clips that come in on loaner drives - i check a few seconds at heads/middle/tails - then DELETE clips from loaner drive -once all clips are deleted i then reformatt loaner drive before sending it back ..

sometimes i edit from external raid drive or if it's a short project i'll transfer to internal drives ...
 
Nick_Shaw said:
"yes I'm happy to erase that RED-DRIVE for re-use".

What do people think?
Click to expand...

"Can I format this drive?"
"Yes you can, it is already downloaded"
------------------NOT--------------------

lost a couple bucks on that one once, anything that will be helpful to prevent this from ever happening again is good input
 
All it would take is a secondary metadata flag on each file on the original drive. Use some RED tool to transfer and verify the footage onto a new drive. When the app determines that the copied footage is intact, it tells the original that it's been duplicated, changing some metaphorical light from red to green. Red light means don't delete me, green light means go for it: I've been copied already.

Obviously it's more complex than that, but such a system could be developed easily.
 
Great topic. This is something that comes up quite often in conversation when we have to deal with P2. Seems like there might be some money to be made if someone can come up with a foolproof system for archiving files. There are options already, but I'm sure that someone can come up with something that really fits into the workflow of these cameras and gives clients a lot of confidence that there footage will always be retrievable.

Steve
 
The way this kind of thing is usually done is by using a file verification algorithm. The most widely adopted is the MD5 checksum verification and I'm sure that would be the way to go in this case.
 
Firstly, I would never trust a single mechanical spinning disk to store my master data. Period. If for god knows what reason that hard drive fails, your screwed. Therefore It would be advisable to set up a redudancy disk array for storage. The chances of one drive screwing up are slim, the chances of 2 are next to zero (unless its a power supply surge. So invest in power surge protectors with battery backups so that the drive heads dont power down abruptly causing damage)

Secondly, as far as as short term storage for rushes etc, there should be no problem using the RED system. Flash memory is highly reliable and im sure the drive array they will be implementing will be thoroughly tested for shock resistance etc..

EDIT: Sorry didnt realise you mentioned redudancy in your post. Redudancy is definately a must. My biggest worry about hard drives is power surges and shock. Those are 2 single reasons for hard drive failure so I hope these issues would be tackled.
 
Last edited:
I couldn't agree more. But really - any kind of storage system that has moving parts are subject to fail at some point. Raid systems and other types of drive arrays are fine, but not really for long-term storage in my opinion.

Who said holographic storage? :)
 
Archive / backup

Archive / backup

A question : How much money are you willing to spend to insure yourself against a potential loss of data - either via catastrophic mechanical failure or operator / assistant error ?

a) In the field

b) In the studio / post suite?
 
tlorenzo said:
I couldn't agree more. But really - any kind of storage system that has moving parts are subject to fail at some point. Raid systems and other types of drive arrays are fine, but not really for long-term storage in my opinion.

Who said holographic storage? :)
Click to expand...

hmm.. long-longterm is a tricky one. Even film has a tough time with longterm unless their stored in salt mine caves at a consistant temprature.

Tape - rots/demagnetises
Hard drives - (lets not go there)
CD/DVD - they also rot unless stored in a vacuum
FLASH - Noway longterm (psychologically speaking, I wouldnt trust data sitting in some virtual space)
Hollerith cards - Now theres a failsafe option!

I think my point is, at the end of the day, nothing is 100%. The only thing you can do is keep your data stored on many multiple sources. If you spread out the copies of data amongst the combination of Bluray, Drive, Tape it shouldnt be an issue.
 
Stuart English said:
A question : How much money are you willing to spend to insure yourself against a potential loss of data - either via catastrophic mechanical failure or operator / assistant error ?

a) In the field

b) In the studio / post suite?
Click to expand...

In the field: If the price meant doubling the cost of the REDRAID for redudancy, that would be fine with me. If the housing of the red raid module needs to be shock resistant (spring supsended drives,whatever) that also should be fine.

Studio: for me, multiple redudancy drives would also fine for that purpose, and the systematic backups to optical or magnetic media.
 
As far as I'm concerned, there should be just as much of a chance of me accidentally deleting my dailies as there would be of me accidentally flashing my dailies. If safety comes down to the user, the user will become good enough.

This comment has nothing to do with physical data loss or redundancy.
 
Brook Willard said:
All it would take is a secondary metadata flag on each file on the original drive. Use some RED tool to transfer and verify the footage onto a new drive. When the app determines that the copied footage is intact, it tells the original that it's been duplicated, changing some metaphorical light from red to green. Red light means don't delete me, green light means go for it: I've been copied already.

Obviously it's more complex than that, but such a system could be developed easily.
Click to expand...

I really like this way of thinking, I see two separate discussions happening here, one is about the safety of the storage mechanisms and durability, the other is to prevent human error or make it really difficult to make such an error.

If you are in a stressed shooting situation you sometimes are forced to pass a device holding image data to a person in between the one that is going to download it, this person will be holding two identical drives at one point in their hands, one to download, and one downloaded to run back to the camera. At that stage it is very easy to make a mistake if you are simply a runner who doesn't know anything about computers or cameras.

A quick glance at the drive would tell you if it is copied, maybe even a way that it knows that the media is actually living on two separate drives, and not until then does the drives light turn to green. At that stage it gets verified and possibly even emptied. Didn't Anders do something like that for P2.

I just don't know if it is even possible to put any light that is not red on a drive from this manufacturer.
 
Stuart English said:
A question : How much money are you willing to spend to insure yourself against a potential loss of data - either via catastrophic mechanical failure or operator / assistant error ?
Click to expand...

In the edit suite I feel confident in my RAID 5 array because of the redundancy built in. I did spend a reasonable amount of money on that array compared to the alternatives, and have never regretted it. I often also back up media onto a Firewire drive and give the client the option to purchase that at the end of the job. This though is based on my current tape based workflow, and I always know that the worst case scenario is I lose time having to re-capture from tape. With a data driven workflow, I feel money spent on redundancy, prefarably storing duplicate copies of media on different formats in different places is money well spent.

During a shoot, I think I personally would rather pay more per RED-DRIVE if they also had redundancy built in. I would be in favour of making them a mirrored pair of drives at the expense of capacity/size. Others may of course disagree. Also the "I've been copied" light on the drive seems like a very good idea.

I have to admit, I have no experience of P2, so others who do are better placed to comment.

Nick
 
You must log in or register to reply here.
Back
Top