Long term/archival storage solutions for digital original media?

[Nick Ravich]

Hello all-

For some reason having a really hard time getting folks to talk to me about this issue so wanted to throw it out to the DVXuser community (not quite sure if this is right spot for it on the forum.)

I work for a small, non-profit arts organization that produces a pretty high profile nat'l documentary series for PBS. We’ve got a growing amount of digital original video material (multi-GB, broadcast-intended digital video files; mostly XDcam EX and P2 original) and we need to get serious about more long term/archival preservation – a system where I can reliably expect to access the media 5/10/20 years down the line. Currently all this media lives on multiple, but non-networked, non-RAIDED external drives; given the life expectancy for these kind of drives, I realize they’re really only a short term solution. Up to the last couple of years, almost all of our original footage was shot to tape; we’ve been creating protection masters, and storing masters and protections in separate climate controlled facilities. Obviously digital material requires a different solution.

One important thing to know about us - we have serious aspirations to preserve all of our originally-produced footage beyond the life of the organization, to eventually make publicly available for researchers, students, etc. So this is not a client-mandated need but instead something generated internally, motivated by our contemporary art and media centered mission. Being smart now about how we ensure the longevity/future usability of this material is crucial for us.

I know the terms "archival" and "long term" probably bring up more questions than answers but I'm wondering how folks in similar positions - smaller production companies producing a consistent (if not broadcaster level volume) of digital original material, who own their media and have a vested interest in preserving it - have dealt with this. Transferring to LTO5 tape? Some kind of cloud/network-based solution? In house? Out-sourced?

Honestly, very surprised there isn't more discussion out there about this. Really hoping I can spark something here.

Sincerely, Nick

[MichaelP]

This is a huge issue and certainly is talked about a lot for those who have high value long term footage than can generate revenue in the future. The issue is that there is no real "long term" archive for 20+ years other than film. Currently LTO is your best bet, but none of the digital archive solutions are benign, meaning there is love and care that needs to happen every so often. If drives, they need to be spun up, need to make sure drivers still exist, need to make sure the files are in an open format as an industry standard in case a proprietary format is no longer available, etc. Even LTO tapes are not guaranteed to play back past two versions - for example, files stored on an LTO-3 tape can be accessed in an LTO-5 capable drive, but not an LTO-2. Also, the tape format used on LTO's is not guaranteed past 14 years. There seems to be a next version up of LTO very 2-3 years. Now maybe some of those LTO issues have changed, but as of 2 years ago, that was what the word of the day was.

But low cost and long term cannot be used in the same sentence for digital file archive once you add in curate costs of the assets to be accessible and readable in 20 years. 15 years ago I edited a feature film where media was stored on Panasonic Phase Change Optical disks (double sided). There is no way I can mount that drive and read those discs today. I would have needed to freeze that Mac system in time - then be able to read them, and only be able to connect them to a SCSI drive which is also a technology on the way out.


Michael

[Nick Ravich]

Michael-
Many thanks for the reply.
Probably a little too glib when I said I had a hard time getting a conversation going about this. Maybe more I wasn't talking to the right people.
Understood about LTO - the best "archival" format, but with serious qualifications. Sounds like what this really means is a on-going commitment to maintaining/up versioning the LTO back ups as time passes and the tech advances. Problematic for us from an organizational/personnel POV but seemingly a commitment worth making.
Curious to hear what/if any experiences you've had with LTO? An in-house solution (ie you've brought in the LTO infrastructure and maintain the workflow)? Or something you've out-sourced? I'm trying to get a sense if this media migration to LTO is DIY-possible or something better left to an out of house IT service.
Also, having a hard time finding "desktop" LTO drives - any reccomendations on where I might look?
Funny you should mention SCSI drives; seems like that's the std LTO hardware connection, no?
Thanks again, Nick.