footage handling on location & archiving footage

Status
Not open for further replies.
I think for narrative productions with a moderate amount of shooting per day, Red Drive will make sense. For those shooting docs, news, etc. with tighter time schedules and higher quantities of footage per day, or low personnel shoots, Red is going to be a big challenge - somewhere in there is the threshold where it won't make sense. IT based solutions are looking to be cheaper overall (no expensive decks required, IT based backup pricing), BUT are looking to definitely be more complicated and manpower intensive.

Under stress and duress, would you rather have something simpler but more expensive, or cheaper but more complicated?

Simpler is going to often win out of necessity for many circumstances.

Keep in mind, Red isn't going to be ideal for every project and circumstance.

In terms of the proposed on set backup and processing - transfer speeds are going to be at least as fast as realtime (otherwise you couldn't have recorded it to that device, no?). Assuming you have a SATA bus on your laptop (think 3rd party card), perhaps copy over to something like another SATA drive or a multi-drive FW800 RAID like a LaCie or G-Tech model.

The other issue of on-set conversion to client friendly format (perhaps native codecs for Avid/FCP/PPro) - I'm inclined to guess that's going to be a slower than realtime process (but I don't know for sure) on a laptop - tough to hand to client at end of day.

Would you clone tapes on set and give one to client? Or just hand to client and it is their problem? These are all problems to be addressed, new workflows to be resolved. If you can copy off to cheap storage (that client is billed for or client provides) and turn it over, case closed - it is their problem to back it up, etc.

However, from my pre-press days back in the 90s, we often had to hnd over large amounts of data to clients. Client provided hard drives were notorious for problems - they wouldn't mount, they would fail and we'd get blamed and be expected to replace the data, or they'd be problematic - they'd have data on them, and gee, can we delete that to put your data on it?

And if we provided them the storage (Zip or Jazz carts back then), getting them back was a nightmare - they'd always hang on to them forever, or lose them, or something. So we had to charge a deposit for the carts, and tracking the billing on them was always a pain because the volume of the job flow was so large.

Then clients complained about getting charged $100 for a $50 item...which we needed to do in order to incentive them to return them to us, otherwise we were their Jazz cart store (a profitless business we didn't want to be in).

If you decide to have a bunch of Red Drives and will hand them to client and expect to get them back....well, you better have a bunch of them - producers will be busy, they won't have gotten it done yet, they'll be nervous about it waiting for a second backup. Or they'll have done that, and haven't had a chance to get it back to you.

Ideally, what we're going to need is a media format that is fast, cheap enough to GIVE to client and charge them a minor/modest fee for it (and media cost needs to be tiny compared to services rendered), and large enough to hold meaningful amounts of data.

I'd interpret that as something that could write data at a rate of at LEAST half that of the recording media, ideally faster than realtime, 50+GB in capacity, not too physically large, inexpensive/ubiquitous readers, a long shelf life, and cost in the $30-$100/hr range or less.

Then again, you can get hard drives for about $0.50/GB. 4K RAW compressed is about 100GB/hr...so a bare hard drive is about $50/hr. Hard drives do fail....from sustained use. Their lifespan is MUCH longer if you just stick them on a shelf, turned off and unused, and don't use them except to pull data. And a Mac Pro has 3 empty slots, you could just socket it in with a power cyle for ghetto cheap access. Or buy a cheap SATA card and hot swap enclosure...or if they client lacks all that, I have a cheap $30 gadget that lets me hook up a bare drive and copy over USB 2 (sloooooooow on Macs, but it works).

While it is great that you can get 3+ hrs of 4K@24p on a 320GB Red Drive...when those get old, they'll suddenly fail, and lose all data on them unless they have some kind of known lifespan or means of establishing viability other than a "Thar She Blows!" indicator of total failure. I could so older and/or abused rental units failing in the field...and unscrupulous rental facilities just reformatting them and sending them back out, hoping they won't fail again...all things to worry about and deal with in the future.

-mike
 
Last edited:
My plan is simply to shoot on the REDDRIVE, archive it to a standard, new SATA 3.5" HDD (320GB models exist and are quite cheap) and give that drive to the client at the end of each day. The cost of the drive is factored into the cost of the hire and they will know what format they will be getting the data on BEFORE I hire my services to them. Additionally (depending on RedCine and its free/cost status) they will get a copy of RedCine to use with their footage OR we will have arranged to process the footage for them (additional cost) into the format they wish to edit with and put that on the drive as well.

For my own projects I'll simply be buying standard HDDs to archive the day's footage on and then ingest the used takes onto my main system project drive until completed. The archive HDD then goes on the shelf not to be touched again unless needed to pull the original footage again for some reason.

Seems a pretty simple solution to me but that is based mostly on a cinema type workflow (my main target).
 
MikeCurtis said:
I think for narrative productions with a moderate amount of shooting per day, Red Drive will make sense. For those shooting docs, news, etc. with tighter time schedules and higher quantities of footage per day, or low personnel shoots, Red is going to be a big challenge - somewhere in there is the threshold where it won't make sense. IT based solutions are looking to be cheaper overall (no expensive decks required, IT based backup pricing), BUT are looking to definitely be more complicated and manpower intensive.

I think I've spent a few too many hours logging and capturing tape and messing with quirky decks, drop-outs, timecode breaks, etc. to agree that tape-based workflow is less complicated and less manpower intensive. (Admittedly some of this was with less reliable consumer tape formats, but still.)

It seems to me that with pretty much any workflow which requires your footage to pass though a computer, you're ahead of the game if you're using a file-based workflow rather than a tape-based workflow, for rather obvious reasons; you've knocked the whole ingesting step out of the workflow. You've either replaced it with a better-than real-time copy operation, or if speed is really essential and you don't mind living a bit dangerously, you've eliminated it completely (you can just edit straight off the digital mag and worry about backups later).

Now, clearly if your footage is going into a pipeline designed around tape and it has to be turned over immediately, you don't really have any option other than shooting straight to tape, but while this isn't unheard of, it also probably isn't very common when you look at the range of jobs for which it's reasonable to use a camera like RED, which is, after all, not exactly optimized for ENG in general.

And, of course, there's nothing stopping you from hooking RED up to an external deck if that's really what you need.

(RED guys, if you really want to drive Sony up a wall, do some reverse engineering and make a portable HDCAM-SR recorder for for $10K. Though, I think you'd probably rather focus on the tapeless future.)

MikeCurtis said:
While it is great that you can get 3+ hrs of 4K@24p on a 320GB Red Drive...when those get old, they'll suddenly fail, and lose all data on them unless they have some kind of known lifespan or means of establishing viability other than a "Thar She Blows!" indicator of total failure. I could so older and/or abused rental units failing in the field...and unscrupulous rental facilities just reformatting them and sending them back out, hoping they won't fail again...all things to worry about and deal with in the future.

Of course, this is not a problem unique to using hard drives in digital magazines, and the industry has come up with a solution: SMART. Practically every drive supports it these days, and I'd really like to encourage RED to have their camera and their digital mags support it! It doesn't catch every failure, of course, but nothing is perfect. Which brings us to....

It's never possible to eliminate all risk. The MTBF of modern hard drive mechanisms can be upwards of a million hours. That's 114 years, for those of you following along at home. Now, clearly not every drive makes it this long (in fact, that MTBF figure means 50% of them don't), and having two drives in a RAID 0 setup cuts your reliability in half... but even so, I wouldn't be at all surprised if, when you ran the numbers, disk-based recording came out to be more reliable than tape or film recording, both of which are fraught with their own hazards.
 
Chris - a very well reasoned argument. I COMPLETELY agree about simpler workflow in post - and as a post guy, I'm all for IT centric. I was specificly addressing issues of production in the field when it needs to go to client.

But even with IT workflow, ingest is cake, conversion fairly straightforward, and the resolution and format options Redcine offers gets more and more valuable the further up the production budget chain you go - EFP/ENG/EPK productions may feel it a slowdown/hassle, but the film folks will love it for their offline/online stuff. And the primary intended market is narrative style productions AFAIK.

Oops, back on topic - but the backup process will be the complicating factor - can't put that XDCAM HD cart or videotape on a shelf as an easy master. The separation of source acquisition DATA from source acquisition MEDIA is the challenge here, and breaks many longstanding traditions of good workflow. Gotta switch gears and go into IT mode, with full paranoia about data integrity the way financial institutions do (but with exponentially more data to wrangle!).

I guess it is most akin to shooting one format and the client wants another - dubbing can be a hassle. Delivering another format, on another piece of media, in less than 3x running time conversion, may well be a challenge.

In my own experience, and what I hear around the web, is that SMART is a "Thar She Blows!" indicator, or sometimes even "Thar She Just Blew!" - not enough warning in advance. Manufacturer's MTBF doesn't seem to connect to my real world experience of 2.5" drives in external devices (laptops) that get toted around the beyond-the-desk world and get dinged around a bit - I've had 3 or 4 friends have their laptop drives die catastrophically with minimal warning in the last year, all on laptops less than 2 years old.

These things do NOT live as long in devices that travel, that seems clear to me from anecdotal evidence. If it sits on a desk and doesn't move? MUCH longer...but that is not the experience these drives will have.

RAID does double the risks, so halves the expected reliable lifetime.

I think having an "hours run" meter on Red Drives would be a vital component.

-mike
 
Last edited:
...and by the way, Blogger has been misbehaving so haven't been able to post for a few days, that's why the silence on hdforindies.com....I'm a victim of their system until they get it fixed.
 
Honestly, I'd much rather use the RED-Flash for most of my shooting (narrative cinema), and as soon as it fills up, dump to an external Firewire 800 drive, verify the copying, dump to ANOTHER Firewire 800 drive as a backup, verify, and then erase the RED-Flash and start over.

Pretty much the same workflow I'd use with P2 cards on an HVX200. Fill up with 24 minutes or so, dump to disk, dump to backup, rinse and repeat.

Of course this workflow isn't ideal for ENG/EFP situations, but for narrative features it's perfect... and no hard drive failures to worry about.
 
Continuing my LTO-3 option, I'd like to remember an idea:

Slating digital footage:

That's what I need:

Something I can use to name the project, 3-8 letters, something pretty straightforward, I don't mind scrolling through the alphabet, just like you did when you got a best score in asteroids. You're only going to to that once a day, maybe less, anyway...

Next I want two numbers to configure, I can scroll them too:
Scene and Shot numbers.
And finally, the Take number.

So in the display, I can see something like this (PS: Loved that the camera has a TC display, so we can leave the monitor/EVF free!):

DVXUSER 002.001 01
00:00:00:00

For a project I named DVXUSER (Just a nickname, of course, not the full project name) Scene 002, Shot 001 Take 01. The time couter is at zero because I didn't run camera.

I want to have three physical buttons I can press. One to change the scene, one to change the shot, and one to confirm, a set button. I would not want to enter a menu everytime there is a scene change. Sure, for dramatic work, what's the deal... But when you're doing docs, having a quick way to keep organized is essential!

If I press the scene button once, the scene number advances one position and starts blinking, like setting up a clock. I can either press enter and leave it one scene ahead, or I can jog and scroll to the number I want. So If I'm running/guning, and see something that I want to tag quickly, I just press scene-enter and I'm in a new scene.

Same deal with shot numbers.

And take numbers automatically go up everytime I cut camera.

That leaves one problem, and that is: what if I accidentally change a scene and want to go back to take 4, instead of shooting take one again? I go to a menu, as punishment for my mistake and worthless being.

When I capture the files, it would be reasonable to expect the files are named accordingly, so For project DVXUSER, I'll find a bin with the files:

DVXUSER_002.001_01
DVXUSER_002.001_02
DVXUSER_002.001_03

And so on.

The main thing is: we have lens controls manually on the lenses. We have decided about format options, camera speed, and shutter speed. We already set our camera LUT so things look pleasant on the client's monitor. And now I'm operating camera for the shot, I want a menu-free life.

I hope I make sense, and I hope other people feel the same way, because I think slating digitally acquired material correctly is absolutely essential, and I just plainly hate menus. Having the information display outside the monitor is a great, necessary idea, so the assistant can check without disturbing the operator. Good job on that!

From my experience as an assistant. Working with Prosumer video cameras in High profile dramatic pieces, "in-set menu navigating" is one of the most terrible experiences one can suffer...

All and all... Here're some pennies...
Thanks for tuning in!

Cheers!

Now that we got that magazine back in the editing suite, there should be a poor soul that is going to view all the takes. He will then fill a lot more metadata slots with stuff like scene description, and why it was or wasn't good. Then he will be able to print a camera report, a table with all that metadata that would read:

DVXUSER_002.001_01 - JARRED CU LOOKING THROUGH GLASS - WEIRD REFLECTION - Zoom, aperture and focus (not electronic, unfortunately, would have to be updated manually) - 24fps - 1/48s - And on set LUT info.

DVXUSER_002.001_02 - JARRED CU LOOKING THROUGH GLASS - GOOD - Zoom, aperture and focus (not electronic, unfortunately, would have to be updated manually) - 24fps - 1/48s - And on set LUT info.

DVXUSER_002.001_03 - JARRED CU LOOKING THROUGH GLASS - BEST - Zoom, aperture and focus (not electronic, unfortunately, would have to be updated manually) - 24fps - 1/48s - And on set LUT info.

And so on. Sure, leaving the system open to a future upgrade where electronic lenses would already give us focus, zoom, and aperture info.
That could be made to work in redcine, so our first ass editor can organize the footage as he is backing up.

Why? Because it helps agilize the editorial process, and since we still haven't really processed our footage with redcine, it could already give us clues to stuff we don't want to mess with anymore (like most bad takes) and save some time.
 
Gun for hire workflow

Gun for hire workflow

I had similar experiences to Mike doing digital pre-press in the mid 90s. Getting drives back from clients in a timely manner is not something you want to count on. My clients are used to being billed for tape stock so my theory is that I will just bill them for FW800 drives, capacity as needed. The bummer will be that running the formatted material out of RedCine via FW800 will be slow. I figure on rotating RedDrives throughout the day, even if they are not full, so I can have the dubbing process ongoing.

Obviously the real world speed of RedCine is an issue in this workflow. I figure I will throw one of the new Mac towers at it. Perhaps it will have an external eSATA port for a faster pipe option???

Blair S. Paulsen
RedOne #19
 
alacritymedia said:
Obviously the real world speed of RedCine is an issue in this workflow. I figure I will throw one of the new Mac towers at it. Perhaps it will have an external eSATA port for a faster pipe option???

FW800 has enough bandwidth that it's not likely to be a significant limiting factor copying data off of a digital mag that consists of a couple of laptop drives in a RAID 0 configuration.
 
More Gun for Hire workflow

More Gun for Hire workflow

I am looking at the transfer speed of the post RedCine data stream which will be more demanding than just copying off the RedDrive. My design target is based on what output format from RedCine I expect clients to request. The goal is to have a system that can crank out that processed footage in a reasonable amount of time. I am postulating that 4:4:4 10 bit, 24p/1080 will be the most likely format - essentially +/- HDCAM-SR level. I am figuring that the time to create and send out this format will depend on how fast RedCine can process a one-light and the transfer speed to the client drive(s).

Are the rest of you thinking that the typical gun for hire client will just want the RedCode RAW? I am operating under the assumption that they will specify a particular format that their Post house/NLE prefers.

Blair S. Paulsen
RedOne #19
 
alacritymedia said:
I am postulating that 4:4:4 10 bit, 24p/1080 will be the most likely format

Does that mean you intend to convert all your rushes to this format on the first pass? That would then only be a one-light style conversion. In order to leverage the benefit of RAW, I expect to just back up the RedCode media (faster than real-time) then do a fast draft quality conversion in RedCine to either DV or DVCProHD, whatever is the required offline format. The full quality 4:4:4 10-bit conversion would then be done at the online stage tweaking the settings on each individual shot in RedCine.

How do other people intend to work?

Nick
 
Personally, I expect that most will work in REDCODE 4K. While 1080 may be popular with ENG shooters and very low-budget productions, the benefits of shooting 4K [RAW!!!] will make it worthwhile. Shooting @27MB/s is just not that hefty, particularly compared to some high-end 1080p formats out there right now.

Needless to say, when I finally start shooting with this camera, 4K REDCODE RAW will be all I need. I rarely overcrank.

Considering that the body includes REDFLASH and that a mini-RAID will be on most people's shopping lists, well, there's redundancy right there. It'll be interesting to see how much an extra flash "magazine" will cost.

As for my later workflow, that all depends on what we seeom from Apple at NAB '07. If I can edit 4K, I will - I have access to enough data bandwidth to handle it. If I have to offline to 1080 or 720 [which is still nothing to bark at], I will.
 
alacritymedia said:
Are the rest of you thinking that the typical gun for hire client will just want the RedCode RAW? I am operating under the assumption that they will specify a particular format that their Post house/NLE prefers.

Assuming RED makes RedCine and Redcode reasonably available to non-owners, I'd think a lot of people would just want the original RAW footage so they'd have maximum flexibility. It's your "digital negative", after all. (And I hope RedCine and Redcode are reasonably available to non-owners. It'll make life a lot easier for everyone.)

It would seem reasonable to charge a fee if you handle the conversion to other formats.
 
Day Rate Profile

Day Rate Profile

Each situation is different but I generally have two modes for my business:

#1 is Project Based - the client needs a particular piece and engages me to make it a reality. I consult on virtually every aspect, hire out specific tasks and do the rest myself. I hire a deck for the final output and manage everything before that as I choose.

#2 is Day Rate - I am hired as a camera op on the clients gear or with my ENG package. At the end of the day or series of days I hand tapes (or in the case of P2, firewire drives) to a producer or client rep.

What I need is a "profile" for myself with the RedOne and kit that I can market to producers to hire me for Day Rate style gigs. Part of the deal will be me making an archive copy (or two) in RedCode onto hard disk and keeping it for 90 days (or more for a fee). If they are sophisticated enough to understand the benefits of the RedCode RAW data set that only makes my life easier. My issue is that I need to appeal to a broader marketplace to keep busy. For many of the clients in my market that are doing HD 4:4:4 10bit 24p/1080 will be the highest format they know how to handle. What I can sell them on is the aura of a "film look" (yes, that may be the most overused phrase in our business) with shallow DoF and due to the oversampling made possible by starting at 4k, an image they can't get with the F900 or Varicam.

I am still concerned that telling some producer that I will be giving them RedCode on a hard drive will elicit blank stares and when they call the post house or editor it will be a thumbs down. Rather than count on the propogation of RedCode into the post community to make my investment pay off I want to leverage RedCine to make the footage I am pimping easy to chew on. I believe that having an NLE friendly format written to hard disk in the field will get me and my RedOne gigs I would otherwise lose to HDCAM and DVCPRO-HD rigs. My pitch will be full raster HD, big 10bit color space, already digitized for your post-production pleasure.

Blair S. Paulsen
RedOne #19
 
ZFS in OS X

ZFS in OS X

Incidentally, this is a fairly interesting development for anyone wondering how they're going to deal with the terabytes of footage a RED One will generate. Basically, while these rumors should always be taken with a grain of salt, it looks like Apple is going to have ZFS support in the next version of OS X, due this spring.

Right now your options for using ZFS are buying ludicrously expensive Sun gear, or downloading OpenSolaris x86 and trying to piece together a system that will work on your own. (I spent some time trying to figure out what hardware would work for an OpenSolaris-based storage server... it's not easy to find good information.)

Having support in OS X would make things a lot easier.

There are some links at the URL above to more info about ZFS, but the short version is, it's a much more flexible approach to managing file systems. You can create storage pools across multiple physical disks, and after that, creating and rearranging file systems is nearly as easy as creating and rearranging directories presently is. Pools themselves are flexible; if you create a pool spanning four drives and need to add a fifth drive, you can do that without having to recreate any of the file systems in the pool. Drives in the pool can be used for any combination of striping, mirroring, or RAID Z (which is sort of like RAID 5), plus there's support for hot spares.

ZFS also has built-in pervasive checksumming features so it'll automatically detect if your data gets corrupted (and recover it, if you've set things up with some amount of redundancy). And because of its architecture, the data on disk is always in a consistent state, eliminating the need for file system repair utilities and the speed hit associated with journaling.

If ZFS support in OS X pans out, you should be able to just hook a couple of port multiplied external SATA enclosures up to a Mac Pro, and have gobs of fast, reliable storage with enterprise class management tools, for about $600/terabyte (raw cost).

For the sake of comparison, an Apple Xserve RAID will cost you ~$1900/TB (and the management tools aren't as flexible), and Sun's storage solutions can cost upwards of ~$6000/TB.

Cheap and easy ZFS sort of does for enterprise-level storage what RED does for 4K acquisition.
 
Since I am an editor rather than a lighting cameraman, I guess my situation may be different from many here.

I reserved my Red because I see the post-production side as being a crucial hub in the data-centric production workflow, and I want to be there from the start. My offering is to production companies, who I will encourage to shoot on my Red, using cameramen with whom I have a relationship and who know how to use the camera properly. I want producers not to have to concern themselves with the mechanics of the conversion and archiving of the footage, as I will deal with all that. I will just assure them that they can have the final film (and any intermediate versions needed) in whatever formats they require.

Where possible I hope to get the producer to budget for me to be on set as a DIT, cloning and converting footage as they go along. But otherwise I would intend to send the camera out with sufficient RedDrives that they would not need to be erased during the shoot, and could be sent back to me in the edit suite. I realise this could mean having quite a lot of drives. Anybody considering setting up a business renting out RedDrives?

Maybe I'm being naive thinking it will be 'that simple'. I know producers can be resistant to change from established patterns of working. But I want to be one of an exclusive group who can, in the early days, say with confidence that I've done the research and testing, and am working with others who have done the same, and we can offer producers the assurances they need.

Nick
 
alacrity - yeah, I think the NLE format will drive the delivery format...just hope that you have all the same codecs they do! If they are asking for some Premiere Pro only format and you're on a Mac, for instance (well, reboot under Boot Camp, then you better have it installed!).

Likely scenario in that instance from post house/client - "Gee, can you just sort of send us a copy of that software and we'll convert it ourselves? Yeah I know it costs $XXX, but we only need it for this one job, so...yeah I know you bought and paid for it, but <subtext>if you want to keep working with us, you'll give it to us for free, so we can use it with anybody's footage we get</subtext>"

Grrr...grrr..grrr...must....release....frustration...with...10 year gone...clients....

Can you guess I don't want to do production work anymore?

-mike
 
SMPTE type standards for codecs anyone?

SMPTE type standards for codecs anyone?

The financial incentive for IT centric solutions throughout the production, post-production and distribution phases is there. Even the largest entities would love to dodge the high purchase and maintainence costs of tape decks. To enable this there must be a set of standards. Perhaps rec. #709 or other such standards already exist and since I am an independent shop and not a broadcaster or distributor I am unaware. I have been told that H.264 at higher bit rates looks amazing.

Could RedCode be a distribution codec? I realize that's a big jump but is the only downside of wavelet codecs the processing horsepower required to uncompress it? Maybe J2K, which is also wavelet based and already in the DCI spec could become a standard. Could we just match one of the tape formats like Panny's D-5 or Sony's HDCAM-SR? I suppose there might be licensing issues but I don't know.

It just seems like uncompressed video is pretty unwieldy when you start talking about full raster 1080 x 1920 @10bits in 24p or 60i (maybe 30psF). I suppose if the disk read speeds don't have to allow real time playback its just a matter of buying bigger drives and then the distrib can copy it onto their balls out server.

Anybody know what's going on with this issue, it sure would make it easier on the indies if there was a set of standards everybody could output to.

Blair S. Paulsen
RedOne #19
 
Status
Not open for further replies.
Back
Top