C500: CRL Data Management

How you deal with backup mainly depends on the type of work you do. I do mostly event so I don't deal with raw or pro res and film in 8 bit H264, but on occasion Prores transcode. The most crucial time to have a backup is the 1-2 week period you're working on your project. I have the original footage in two locations, the rest of the project is kept separate and is backed up every day or when significant changes are made. After the project is completed, everything is moved to a mirror raid spinner, then after a year all the original footage is deleted, only the final export and supporting files are kept. Even that takes up a lot of space. I've archived on large spinners but the reality is if they're not raided you could loose everything if the drive fails. I almost never have someone come back for edits but I occasionally dip back for samples of work, client highlights, or using effects or resources such as music from previous projects. I do have a very nice client who I filmed a week for a special project that is complete but wants to re-edit at some later date. That has meant I've had to hold 1tb of video for a couple of years. I don't like projects that drag on because the longer they remain your risk of hardware failure increases.
 
Last edited:
ahalpert said:
These sound too good to be true, but PremiumBeat just put out a blog post about footage archival cloud services and if I'm reading this correctly, something like CrashPlan will back up unlimited footage for you for $10/month: https://www.premiumbeat.com/blog/im...-08-16&pl=emailcore-shutters.22831625&cr=null
Click to expand...

I'll never do that again.

1st I uploaded about 12TB to Amazon Cloud unlimited (60€/y) when they sent an email they changed the business model and they cancel my subscription. New price after 180 days 60€ per TB. Thanks but no thanks.

at that time with Backblaze all your media you store in the cloud has to be connected once a month. If you have it on external drives you have to attach it once a month or it will be deleted from the cloud. I don't know how it is today?

Then I went for crashplan and after a while I realized that the cloud backup was way behind. After several support cases they admitted that they lowered the bandwidth to 30GB upload per day. Means max 1TB per month. I cancelled that too.

I don't want to be dependent again from those changing business models.
 
Yikes, Clermond! So it was too good to be true.

And I just checked Dropbox again - they used to offer an unlimited storage plan but they no longer do.

Personally, I keep a backup of all footage and project files for about 5 years. Less if they're no longer a client. Longer if they're a big client.

I haven't had to buy storage drives for a few years because I have enough by now that by the time I need more space, my oldest archives can be recycled.

But RAW files are about 5-10x the bitrates of the footage I've been archiving. Of course, HDDs are now about 1/3 the price of when I built my HDD collection, so it might be financially viable to build a large enough collection of drives to back everything up.
 
Maintaining your own raid is most sensible for large amounts of data. Thing that stops me from a really robust Raid 5 solution is the high up front cost and derived benefit. IE clients aren't paying me to back up their videos. For final exports, not a problem but raw footage the costs is too much. 5x $200 for larger spinners, then $800+ for the raid system, you're looking at $1,800+ for something that doesn't make you money. If you use the raid to edit off of then the cost can be justified.
 
Last edited:
Just this past year I transitioned over to a RAID instead of the JBOD method I had been using since getting into the business many, many moons ago. The motivation was exactly what you describe earlier...a spin drive failed and luckily it wasn’t anything I was in need of but it really drove home the need for something more trustworthy.

It was a bitter pill to swallow for sure but I do edit off the raid and the resulting bump in speed has been nice...it’s not nearly as fast as reviewers on YouTube try to make you believe but it’s definitely faster than what is possible on a single, rotational drive.
 
I back up everything to 2 matching drives when it's time to archive it and just do it manually. Does that make me a caveman?

As Norbro pointed out, 8TB drives can be purchased on Amazon for under $150 nowadays. So you can archive 8TB redundantly for under $300 and eventually it will be recycled (if you're me)
 
I genuinely had no idea people still use spinning drives for editing (like I actually thought most were discontinued, the internal ones without a casing)...but for external storage, so worth it.
 
ahalpert said:
I back up everything to 2 matching drives when it's time to archive it and just do it manually. Does that make me a caveman?

As Norbro pointed out, 8TB drives can be purchased on Amazon for under $150 nowadays. So you can archive 8TB redundantly for under $300 and eventually it will be recycled (if you're me)
Click to expand...

I don’t think it does; essentially that’s what I do. When it’s time to “archive” for either a one-off production or a client I am no longer working with I either offload to a bare drive or, prior to raid setup, I would just pull the JBOD drive out and then put that archive drive away. I just have too many different projects, clients or just storage in general to leave everything connected to system all the time. I don’t know if many people who can.
 
digitalinnovations said:
. I just have too many different projects, clients or just storage in general to leave everything connected to system all the time. I don’t know if many people who can.
Click to expand...

Oh for sure. I have a box of HDDs sitting in the closet that I periodically add to and pull from. God help me when I actually have to locate something in the archive from a few years back (which seldom happens). but at least it's there
 
NorBro said:
I genuinely had no idea people still use spinning drives for editing (like I actually thought most were discontinued, the internal ones without a casing)...but for external storage, so worth it.
Click to expand...

Haha! I’m pretty sure all / most system drives are now some form of solid state for the speed necessary for apps to run smoothly. I’ve never used the system drive to edit from but it may be possible with the new, fast storage they have - the issue would just be capacity I suppose. I’ve only ever used spinning drives to edit but they’ve always been in external enclosures or on a separate bus internally in the case of the larger, tower computers that have room for additional internal drives.
 
ahalpert said:
...God help me when I actually have to locate something in the archive from a few years back...
Click to expand...

I actually had a client not too long ago ask me if I had the footage from a shoot / project that was like 12 years old...and I actually had it, which is crazy. It ended up being something we shot in HD and delivered in SD so I was actually able to provide a higher quality deliverable than we originally did. So weird.
 
ahalpert said:
Oh for sure. I have a box of HDDs sitting in the closet that I periodically add to and pull from. God help me when I actually have to locate something in the archive from a few years back (which seldom happens). but at least it's there
Click to expand...
That's the point of a raid. Everything is in one location. When you archive a project you copy once and it's automatically backed up, if you make a change there's no need to make sure the other drive is up to date.
NorBro said:
I genuinely had no idea people still use spinning drives for editing (like I actually thought most were discontinued, the internal ones without a casing)...but for external storage, so worth it.
Click to expand...
The other point of a raid is that it becomes faster as more drives are added. This is because 5+ drives can be writing/reading at the same time. Generally speaking people edit raw would still use this solution if you have large files to deal with (above 2tb).
 
Last edited:
digitalinnovations said:
I actually had a client not too long ago ask me if I had the footage from a shoot / project that was like 12 years old...and I actually had it, which is crazy. It ended up being something we shot in HD and delivered in SD so I was actually able to provide a higher quality deliverable than we originally did. So weird.
Click to expand...

wow that's awesome. i love it when things like that work out. and you just became their MVP
 
Peter C. said:
That's the point of a raid. Everything is in one location. When you archive a project you copy once and it's automatically backed up, if you make a change there's no need to make sure the other drive is up to date.
Click to expand...

yes of course. the only point of buying individual HDDs is that they're dirt cheap. but if the footage is all backed up and you only need to update the project files, it's not too cumbersome
 
Peter C. said:
The other point of a raid is that it becomes faster as more drives are added. This is because 5+ drives can be writing/reading at the same time. Generally speaking people edit raw would still use this solution if you have large files to deal with (above 2tb).
Click to expand...

I definitely hear you but that's why I started using SSDs to edit 10 years ago.

In 2020, plug in one 4TB SSD for $300 with a $15 USB adapter and you're still faster than many HDD RAIDs (at least on our level, consumer/prosumer).
 
Original OG 480GB...was like gold to Blackmagic BMCC/BMPC users as it was THE drive to use, especially after the updated versions caused issues. (Eventually more and others were introduced to the market.)

This one had about 600-700 projects/edits on it in its life since 2012.

SSD.jpg
 
Peter C. said:
That's the point of a raid. Everything is in one location. When you archive a project you copy once and it's automatically backed up, if you make a change there's no need to make sure the other drive is up to date.

The other point of a raid is that it becomes faster as more drives are added. This is because 5+ drives can be writing/reading at the same time. Generally speaking people edit raw would still use this solution if you have large files to deal with (above 2tb).
Click to expand...

I think that’s JBOD not a raid he was taking about.

Raid for editing for sure but topic was backup where no raid is needed. I was thinking about a LTO but I’m not convinced that it’s future proof in terms of software for read/write.
 
Clermond said:
I think that’s JBOD not a raid he was taking about.

Raid for editing for sure but topic was backup where no raid is needed. I was thinking about a LTO but I’m not convinced that it’s future proof in terms of software for read/write.
Click to expand...
My point is if you use the raid to edit you can also use it for archive. JBOD is a raid, there are many types of raids, and they can be built to be large and fast that's what servers that run the internet use. For the OP raid is the most logical solution and is more efficient use of hard drives space, because backing up on separate drives requires double the space where as raids does it in such a way you get more space.
 
Last edited:
Peter C. said:
My point is if you use the raid to edit you can also use it for archive. JBOD is a raid, there are many types of raids, and they can be built to be large and fast that's what servers that run the internet use. For the OP raid is the most logical solution and is more efficient use of hard drives space, because backing up on separate drives requires double the space where as raids does it in such a way you get more space.
Click to expand...

not at all.
With a JOBD you need one rack or even a dock is emough (that's what I have)
What do you do when the RAID is full? Buy a new one? The case is a grand alone and a dock is a tenth of that.
You don't need speed for backup. Let CarbonCopyCloner do the job when you do sth else.

And one back up isn't just safe no matter if it's a raid.
You should backup your editing raid like with a time machine concept too

Peter C. said:
JBOD is a raid
Click to expand...

by the way: no, a JBOD is a bundle of disks. it is not a raid.
 
Last edited:
You must log in or register to reply here.
Back
Top