Storing footage shot on hvx and timecode issues

J

J5 squared

Well-known member
So i was just hired as an assistant editor at a shop in town. They've been shooting with an hvx for a few months and the system they have for storing the footage seems some what ludacris to me. At current moment footage is shot and then loaded onto a laptop with external drive in the field. They then bring that drive to me and want me to take the clips onto a timeline and print it to tape. They want it on tape for two reasons, A) its too expensive and not safe enough to store it on harddrives forever B) they want the stored timecodes to match the timecodes used in the edits. So if they want to go and reassemble projects or find shots later on they know exactly where that shot is on the original tape. But that means that after i print it to a tape i have to delete all the footage from the drive AND THEN go back through and capture all the clips from tape again so that those clips have timecodes that match the tapes. So it goes like this, camera to computer, to timeline, to tape, to computer. So when you think about the time involved, if you have an hr of footage you have to lay it all to a timeline so theres time spent there. Then you have to print it to tape so an hr of footage takes an hr to put to tape. Then you have to go back through and log all the clips again so theres time spent there, then u have to capture all the footage which if you have an hr of tape takes atleast an hr to capture. To say the least after doing this all day it gets rather old, but they haven't been able to come up with a better solution. I just started running the problem through my head today and so far haven't thought of anything. What is everyone else doing for storage? Are you approaching the timecode issue any different than we've thought of? Can anyone recommend any better less time consuming method?
 
This is the workflow I'm going to employ in my setup for now:

-- MXF files are copied from the Firestore to a mirrored RAID, either in the field (if I'm travelling) or in the edit bay.
-- Import to the edit system (Final Cut Pro) and set aside the MXF files which are permanently stored on the mirrored RAID.
-- After the program is finished, create a "Media-managed" version with only media that was used in the program to save space.

If the media from the media-managed version gets messed up for some reason, it's possible to re-import from the MXF files and re-connect clips in the media-managed program to the newly imported clip. I tested this and it seems to link up just fine, even though there's a warning stating that the start and end points of the new media doesn't match the clip.

Of course you will need to know which clip connected to the original media, and that's why it's important to make sure each clip retains its reference to the original media to let you know where that particular clip came from.

The edited clips in the timeline or sequence of a program will, in fact, connect up to the proper point along a piece of media that was re-imported from the original MXF file.

So, while it's not possible to "recapture" as you would with a program on tape, it's still possible to re-import the original media and reconnect the clips in a program as needed.

This can eliminate the reliance on tape and the problems that will incur.
 
Yikes what a workflow...

Yikes what a workflow...

That is pretty harsh on your end. But unless they can get enough cards that they don't need to dump materials off in the field you're kind of stuck. If they give you the cards you can run that w/ the shot timecode to an AJ-1200 and you're golden (well sort of anyway). But it doesn't seem like that's going to be much of an option for you.

The one work around, but might be a tough sell on your part, is to see if they'll let you burn the MOV files to DVD-R's. If you convert the mxf's to mov's and confirm the mov's are sound, then there's nothing wrong workflow wise putting them onto DVD-Rs and that will preserve TC for you. If they complain about reliability issue, tape degrades, even in cold storage. DVD-Rs degrade too even in cold storage. So really you're at an equivalent level of backup and improve your efficiency. Problem is selling it too them.

Oh actually, come to think of it, there could be one other way, but this is purely hypothetical and the DP has the camera so I can't test this right now. But if they simply copy over the P2 folders exactly as is, not processing it in P2 Genie, P2 Log, or ANY software, if the folders are on the HDD are the true raw images, then how about this:

Have them leave you the camera over night, while they are not shooting. Connect the hard drive and the camera to your computer. With all drives mounted, one card at a time copy the RAW folders back to the cards in the camera. Once you have your two cards (or perhaps just one card if they are resetting TC after each card) loaded back up, connect the camera to your AJ-1200. Now you can copy the footage to a tape with the camera shot timecode. So when you eventually load the stuff into FCP for example, the TC's on the clips and the TC's on the tape will be the same.

There is one fudge here though. If a clip was shot across two P2 media cards it reads the clip as 1 solid clip with continous TC in camera and will send to your VTR like that. However, it will be 2 clips when you import to your NLE, and the TC won't match perfectly. So if you were to go that route, and they agree to go that route, you need to have them only put 1 card in the camera at all times and just swap more often. A bit of a pain on there end, but might make your post workflow a lot more tolerable.

Okay well that's my best stab at it. I gotta get back to work here. Hope that helps or at least maybe gives you inspiration to figure out a workable solution on your own.

cheers.
 
How would you convert them to mov's? Don't you have to bring them into FCP and then render out as a mov, and at that point wouldn't you have lost all the time code from the original clips and then have adopted the fcp timeline TC? Taking each clip and converting it individually to mov in FCP would be just about as tedious as doing batches to and from tape it seems.

We have an aj-1200 so your other idea is interesting. So if we hook up the output of the hvx to the aja and then his play and record on our deck it will preserve the timecode? Will it bounce all over and keep the timecode in sync? Is there anything special that needs to be done to the hvx, hook somethine certain up so the deck knows the timecode? Someone needs to write a programe that can read the raw p2 format and play it back on our computer that would be a little easier, but i suppose in honest whenever they are back to give me the drives the camera is there as well.
 
MXF files (from P2 or Firestore) are imported and converted to Quicktime files in FCP. There's no rendering required afterward except for effects. And the timecode remains intact.

You have the option of importing a single clip or a selection of clips, so it's not just one clip at a time. And it imports in less than real time.

Immediately after it's imported, you're editing. Much, much less tedious than tape. With third party software, it's possible to import only selected portions of clips, too.

In my opinion, if you're working tapeless, then it's best to configure a workflow with that in mind. Going to tape at any stage of the process has to be done in real time, whereas moving files is almost always less than real time. There's also little or no generational loss and less likelihood of problems such as dropouts because file duplication usually involves some sort of error checking process -- which may not be available in the tape duplication processes.

There's always the possibility of a drive failure, but that's what backups are for. And a mirrored RAID provides an instant backup. That's why that's the first stage of my system as well as its archive for source footage.

Also, my RAIDS are all set up with hot-swappable drives. That means a 1-terabyte system can be readily configured for individual projects. Each producer gets a set of hard drives for their project -- for archive and for editing.

And as for cost: a minute of hard drive space on a SATA drive is about the same as a minute of DVCProHD tape. Except you don't have to buy and service a $25,000 deck. Also, it's a lot easier to locate and retrieve something off a hard drive than it is off tape.
 
I agree with you sensui, trying to stay tapeless is the way to go. Asides the money they waste paying me to lay off to tape and capture again, those tapes are also expensive. And believe me i burn through them pretty fast laying all this off. I musta been a bit tired yesterday in not thinkin of how the p2 raw gets into FCP and becomes .mov saving those .mov's to a dvd after they've been renamed seems like it would work to me. You'll still have all the clip names and timescodes to match up against your edit. The only other thing i can think of is that when i'm laying to tape i'm also monitoring the deck with a dvd recorder and making a dvd of the session. So we have a dvd someone can plop in a player and watch with the timecode running at the bottom then when they see a shot or something they want they just mark the time. How could you make something like this for the .mov files? The only thing i can think of is like a batch watch, you'd just have to say "you have to watch this on a computer not a dvd player" then they just record the clip name and timecode of the one they want. To me thats still a better idea than going through all the trouble of messing with tape just to have a dvd. But in case anyone has their head on straighter than me, can you think of a way to make a dvd out of those original .mov's where you'd still have an accurate timecode with each clip displayed?
 
J5...

Not sure how other NLE's work, but in FCP there's a timecode reader filter that can be dropped onto a clip or onto an entire sequence. It will provide a window burn of the current timecode from either a particular clip or from within an entire sequence.

It's also possible to combine these with one window giving the sequence timecode and another providing each clip's timecode.

This may require rendering, however. And it'll also have to be exported from the sequence and authored into a DVD. Both can be time consuming. But in my case I do it during off-hours and let the computer handle the rendering and encoding when I'm not using it.
 
DCSensui said:
J5...

Not sure how other NLE's work, but in FCP there's a timecode reader filter that can be dropped onto a clip or onto an entire sequence. It will provide a window burn of the current timecode from either a particular clip or from within an entire sequence.

It's also possible to combine these with one window giving the sequence timecode and another providing each clip's timecode.

This may require rendering, however. And it'll also have to be exported from the sequence and authored into a DVD. Both can be time consuming. But in my case I do it during off-hours and let the computer handle the rendering and encoding when I'm not using it.
Click to expand...

well i like the sound of that sensui. I'm a vegas dvx user with my own stuff so its taking me a bit to get used to their FCP and hvx hardware. I'm going to mess with it a bit when i get the chance and try and just show them what could be done. To me that makes alot more sense.
 
Alright so i think i'm close to having a viable setup of going straight from p2 into FCP and making the clips .mov's. Then laying them to a timeline with a "timecode reader filter" so we know the time from each clip. Now all i need is something like the timecode reader that will display the clip name along with the timecode. Since the tc won't run the whole dvd you'll need somethin to identify each clip, otherwise you'll have a ton of tc's and won't know which is which, so a tc reference won't help. But if they can watch the dvd and say clip 11, or headshot1 at 3:49 then i'll be set. Anyone know of something i can just apply to all tracks that will display the clip names like the tc filter works? It would be possible to jus go in and add txt to the clips but this will add more time and i'm tryin to cut as much time from the processes as possible.
 
J5, I feel your pain, I'm shooting a feature right now and just went through the processes of convincing the producer, who owns a very large post house in LA that he didn't need to bump everything to HDCAM and that he would actually yield a better final product by not going to tape. The system works perfectly and in my opinion securely without needing tape at any point.

Here is how I do it:

10 Easy if Long Winded Steps to P2 Bliss

1) Record to 3 4G p2 cards in the field with the camera in Record Run TC. I never reset the TC so I have a running time code across the entire shoot, unlike tape where I would reset the TC to help identify the tape. With P2 this ID is created elsewhere.

2) Copy card to the P2 Store, reformat and cycle back into the camera. This could also be done using a laptop and external drive of your choice. If using an external drive, it is best to copy to two drives for security. The p2 store is the only single drive that I trust to hold otherwise non-secure data.

3) At the end of the day, dump the P2 store to your long term storage drives, I use inexpensive raw 3.5" hard drives in the smallest capacity I can find. The idea is to minimize the amout of loss if one of these drives fail. You can find raw hard drives in bulk for about $.65/Gb much cheaper than the equivilent in DVCPro Tape or HDCam. Seagate, Maxtor, and Hitachi have served me well. I mount these in a small 2 slot FW800 tray based raid like this http://www.directron.com/venusds3r.html

4) Copy each card to it's own folder named after the modification date of the LastClip.txt file. This file is the last thing written to the card and is an accurate indication of the date and 24 hr time the card was recorded. This ID will become the Reel ID in FCP that will help you find the footage should you need to reimport. Think of it as a replacement for the Tape Number when digitizing. My Card IDs look like: Project_Name_060719_1532 . P2 Genie has the ability to create this ID for you (Thanks again Anders) and I;ve found that this information is a great way to prove days of work when invoicing.

5) Burn a DVD of the card using the Card ID as the name of the disk and write the ID on the disk itself. While the DVD is burning, switch over the PhotoShop and open the Contact Sheet automator under the file menu. Choose the ICON directory of the P2 directory that is currently burning. PS will create a page of icons with the clip names printed below the icons. Add a text layer with your Card ID at the top of the page and print the sheet. Fold it in quarters and slip the DVD in the print out. Store the DVD in a CD case for the project.

6) Import the footage from your raw hard drive into FCP with your capture scratch set to a RAID of sufficient size and speed to handle the needs of your project. The top of the P2 Import dialog has a text field called Reel Name that defaults to "P2 Import". Type in the Card ID to match the folder name that appears in the Volumes/Path pull down just below the Reel Name text field. Select Import All at the bottom Right of the dialog box.

7) Sort your bin by Media Start and you wil have all of your clips in the order that they were shot, including clips that spanned mulitple cards. Select all of your clips in the bin and drag them to the timeline. Select all clips in your timeline and click on the Time Code Reader filter under the Video Filter Menu. This will give you a window burn of the origional camera Time Code for every clip on the timeline. Export this timeline to a low res QT or to DVD for producer and director viewing.

8) Rename the clip names to a meaningful description. Do not worry about saving the clip names. FCP remembers where the clip came from. If you ever need to know the origional camera generated clip name, simply [Control] click on the clip in the browser and choose Item Properties at the bottom of the menu that will appear. Click on Format and you will see all of the information about your clip. The 4th item down is "Source" and it contains the directory on your capture scratch where the .mov file lives, it will be named the same as your camera origional clip name. Click on the Logging Tab in the Item Properties dialog, and the second item will be Reel. This shows the Card ID that you typed in on the P2 Import dialog. This information remains even if the media file is deleted. you could crash your entire raid and so long as you have a backup of your project file, these two pieces of information will allow you find the drive or DVD containing your origional MXF data for remport and relinking.

9) Pack the raw drives with your MXF data away in static safe bags and padded boxes for safekeeping. I find that drives fail mostly from heat and use, so storing them for long term storage seems at least as safe as storing tape. At least the magnetic media in a HDD is hermetically sealed in a dust proof enclosure, unlike tape.

10) If you still feel the need for a tape archive above and beyond the DVD and HDD backups, consolidate your final edit and write the media and project files to DLT or LTO tape.

Hope this example will help you convince the producers to leave tape behind. My writing style probably makes this seem overly complex but it really is a straight forward and simple process once you've gone through it for a couple of cards.

Best of luck and PM me if you need any help decifering my writing.

-filmstox
 
Last edited:
Damn, got a little too caught up with a deadline to keep following this post, seems to have gone ahead on me quite a bit.

Someone asked if when you firewire through to tape in the AJ-1200 do you keep the timecode, and the answer to that is yes you do. However, if you have shot 24pN then it will be slightly different on tape (it'll like you had shot with the Varicam). So really doing that is good for 'emergency backup' or for say stock footage archive where someone else is going to be paying for the footage and can log and load it themself anyway. So perhaps not for everyone, buts working well enough as an archival choice for our purposes.

As for the whole timecode issue for making DVD burns, well just do go out of the RCA jacks into your DVD Recorder and adjust the necessary display output settings and it'll give you a cheap and easy timecode burn right there. It does come out little stretched (cause you are going HD to SD, and it doesn't have a great convertor for that), but its good enough for the director to watch for his/her notes or paper edits. If a producer is watching for quality purposes, well okay then that might not work (depending on them). Forget which menu settings it is though and as the DP has the camera in the field right now I can't look it up.

cheers.
 
Hi there filmstox,

Thanks so much for the "10 Easy Steps"! I am going through them right now as I am trying to master the best way to organize and manage my media. I have a question about number 4. You say to copy each card to it's own folder. Do you mean copy it to another drive and name it using the modification date and then therefore essentially having two versions of the card folder. One not renamed on the original Raw HDD and another renamed and located on another HDD that will be used to then import into FCP? (This will also answer my question for number 6 which is if there are two places and names for the card folder, which one should I be bringing into FCP).
Again, thanks for your very helpful post.

--angela
 
angela said:
...one not renamed on the original Raw HDD and another renamed and located on another HDD that will be used to then import into FCP?
Click to expand...
Angela,

Glad the steps are useful. You will import the footage from the raw drive. FCP copies the footage to your scratch drive (RAID) as part of the import p2 process. You will end up with a Capture Scratch folder full of .mov files named after the camera originated files. After import, pull the raw drive out and store it for safe keeping. The second raw drive (if you aren't using the p2 store) can then be reformatted and used again as the backup in the field.

The modified date naming convention should result in a set of files, all named the same. At the end, you should have:
1 Dvd-r labeled like Project_060725_1054 where the date and time are the modified date of the lastclip.txt file from the original p2 card.

1 Raw drive full of folders named as above each containing the contents of 1 card.

A final cut project with all of your clips linking back to the folder name on the raw drive as it's source. And all of the data converted to quicktime in a capture scratch on your RAID.

I hope that didn't make it more confusing.
-filmstox
 
Last edited:
Filmstox,

Thanks! That clears it up for me. I appreciate your quick response.

The steps are very useful.

-angela
 
You must log in or register to reply here.
Back
Top