HPX250: Making a P2 "Library" and the story of how I got nowhere.

Unfortunately Jan, your post put the final nail in the coffin for our P2 times around here. The product from Imagine that you refer to is not a pro tool and should not be recommended in this case.... We have very much enjoyed working with the P2 format over the years, but the time has come to go the S-way as we find ourselves in the exact same position as the poor person starting this very important thread. Building a solid, easy to use, and simple to build footage library with P2, no good....
We will miss our great P2 camcorders, I would still recommend these great cameras for single smallish projects, but for serious TV-production work and the obvious decision off creating a library that all involved can browse through...sorry..
 
Most smaller, local news stations don't archive their raw camera news footage. Pretty much discard them after 6-9 months at most.
It's very costly to keep everything archived. These files grow pretty large. But some stations will require to archive the final on-air broadcast.

What's still perplexing is the move to all solid state acquisition like P2 but the archival format is on LTO-5 tape format!
I'll have to spend about $2K on tape drive that'll wear down over time just like a VTR and tape media costs.
It appears that tapeless workflow is actually not tapeless at all.
 
Don't give up the Axlevideo.com new software coming out at IBC may do the trick. Is P2CMS totally out of it? I do not understand it and neither does anyone in Panasonic I suspect. Still waiting for call from Product Manager. Support have wiped their hands of it.
 
saltflyer said:
Unfortunately Jan, your post put the final nail in the coffin for our P2 times around here. The product from Imagine that you refer to is not a pro tool and should not be recommended in this case....
Click to expand...

Hey Saltflyer, Well I think the MacClipLibrary is a pro tool and there are a good number of large companies using it, but if you think not, there are others. Browsing file based footage is based on looking at the files and being able to find them. This is and can only be done in the computer domain. This is done with meta data and it really doesn't matter whether it is Panasonic or Sony, you have to go through the work of establishing how you are going to manage the data. The news folks use servers and meta data and trust me, they end up with a lot more footage than they want to deal with as well.


saltflyer said:
We have very much enjoyed working with the P2 format over the years, but the time has come to go the S-way as we find ourselves in the exact same position as the poor person starting this very important thread. Building a solid, easy to use, and simple to build footage library with P2, no good....
Click to expand...


There is a way to work with it and it is simple, but you can't look at it 3 years into shooting and say wow how do I deal with this and write it off because you have drives full of video and the work of moving to a library system seems overwhelming. It would seem that way, whether you are on P2, AVCHD, or XDCAM SxS. File based workflow has to be structured from the very beginning into a library of sorts, and leaving it on hard drives is one way, but I don't think that is the best. A good number of the guys that I work with leave it on the HDD for an interim period, say for about 6 months after the client accepts and then move it onto an LTO drive. They then shelve that tape with the client's project name on the outside. Similar to they way they used tape, but vastly cheaper. Other guys, log it all with the metadata so that the DataBase will help them find where the shot is that they archived 4 years ago. It really depends on the nature of the video you shoot, but the process is the same regardless of manufacture. If you are saying that you are going to go with the disk based system, then your archive becomes the same as the tape based workflow and just because you have a disk does not mean you have an archive, you only have a disk and unless you establish how you are going to organize it, categorize it and retrieve it, you do not have a library.

saltflyer said:
We will miss our great P2 camcorders, I would still recommend these great cameras for single smallish projects, but for serious TV-production work and the obvious decision off creating a library that all involved can browse through...sorry..
Click to expand...

So I ask, how are you going to browse through any library of data, without some work of setting up the structure. In and of itself, a "library" requires structure, that has an overview of a database for finding things easily. So from your comment, it sounds as though you think other file based systems are going to set it up for you? I think not, it doesn't happen without some work on the front end, sort of a plan your work, work your plan.

Best of luck to you,

Jan
 
P2CMS is a little limited for archival work. It can only look at the HDD it is attached to, so unless that HDD is huge, I mean really huge, your archive will not be very large. It, I believe, was written for the producer that works on a drive by drive basis, catalogs the drive and puts it on the shelf.

Hope that helps,

Jan
 
Thanks for your input Jan. I will continue to look for the Holy Grail. We will be setting up a P2 based + other MXF media,Nano etc Library from scratch.
 
Wow, this thread has expanded a lot. Thank you all so much for your input. However I feel things may have been lost in translation, you see the whole process of my library was not to just "archive" everything away for another day. It was actually to delete most of it clearing up space for us...However at the same time taking the best clips and storing them away for our editor should the need arise too go back to earlier episodes. But the P2 Format doesn't let me take bits and pieces except for P2CMS but even then you don't just have a "clip" you have a mess of folders that don't work well when using a search engine.

From what Jan has said I gather we should have been doing this library of sorts from the beginning, but why is it so hard a little bit down the track? I've built a library, structure, search engine and all on a raid drive. All I want is to be able to take certain clips, move them to their place in the library and then clean out all the rest freeing up the hard drives.

But if I was to move to Xdcam using .mxf files, with the ability to individualize each clip and view with XdcamBrowser wouldn't that solve my problem? I would be able to pick and choose clips and place them into the library, I would be able to delete all unwanted footage and and the editor would be able to "preview" clips before taking them to Premiere.

So my TL;DR for this is how do you chop up P2 footage to make individual clips, retaining all audio and video, without the mess of folders and crap P2CMS produces when it rips them out.

This isn't about "Storage". This is about the ability to take what I want, order it into an easy and ready to use library and deleting all left over footage that will no longer be required. The editor then has the ability to search for a shot, coming to an individual clip that he can then "preview" and import it to premiere.

Thanks everyone though, I really appreciate all the help I can get.

Cheers,
Tom
 
Exactly same situation here....
My question is the same: Will XDcam produce the same problem or will building a user friendly library be an easy task??
 
I saw at NAB Sony has a very good solution for flexible asset management. Forget the name. But the price tag is another zero to the end and an assortment of XDCAM decks, support fees. It's more geared big budget broadcasters or high end post houses that can afford six figures or more. For many of us, I rather wait until something more cost effective.
 
I think if you take a look at the Media Clip Library from Imagine, that offers the ability to store which clips you want and to be able to retrieve the clips you want. They have an online demo and a phone call to Dan Montgomery will clear up and additional questions that may come up. It is a fairly easy system and can grow fairly large.

And any solid state system will present the same issues. The better ones will allow you to search by metadata. Like finding a clip that you tagged by name, say Empire State Building or ESB.

There are others and I am sure that at IBC another group of them will roll out as Digital Asset Management is key at this time. Your cost will vary as your needs vary.

Best,

Jan
 
ggrantly said:
The point is that even if you need something else for your editor du-jour, even if using windoz, prores will not be difficult to deal with. What you need is a reliable format to archive to and prores is pretty universally known and accepted. Another codec could be used also. Compare this to storage in the native codecs, which at this point are changing faster than editing codecs. Capture codecs are rarely used for editing excepts for maybe the news.

Side note: considering how Avid is bleeding red and shrinking by the year, do you feel more comfortable hitching up to Avids wagon more than Apple?

Grant
!
Click to expand...

You're way behind the times. Windoz was used 20 years ago, but doesn't apply today. All Hollywood movies are authored on PCs with software that Mac doesn't have. And, I could go on, but... I have both. "Capture Codec" For last 5 years, I have archived P2 on harddrives with an Excel printout. And, for that many years, have edited native P2 on the Edius timeline, containing many formats. Why waste time converting to ProRez, which has a much larger size than P2. Anyway, I love the new P2 contained AVC-Intra 100 that I'm shooting with my HPX250, and editing in 10 bit realtime. Life is good.
 
Last edited:
JohnnyD said:
You're way behind the times. Windoz was used 20 years ago, but doesn't apply today. All Hollywood movies are authored on PCs with software that Mac doesn't have. And, I could go on, but... I have both. "Capture Codec" For last 5 years, I have archived P2 on harddrives with an Excel printout. And, for that many years, have edited native P2 on the Edius timeline, containing many formats. Why waste time converting to ProRez, which has a much larger size than P2. Anyway, I love the new P2 contained AVC-Intra 100 that I'm shooting with my HPX250, and editing in 10 bit realtime. Life is good.
Click to expand...

Not sure about the windoz comment....I meant generically the microsoft PC platform, I mean you might still be using Vista.....Just kidding. I don't do "Hollywood movies" , so whatever your perception of the shortcomings of the Apple platform don't much matter to me, aren't we discussing archive strategies for smaller producers? My point was to archive in a consistent format which is universal and ProRes is certainly that, though if you are totally PC based, another codec without the need for encoding would be better. Drive space is pretty cheap but admittedly the higher bitrate ProRes formats do take up lots of space, but those aren't always needed depending on the capture codec. I don't think there is much doubt that a strong suit of FCPX is its metadata ability. If you don't like ProRes or anything Apple, pick another. The OP was interested in a archival scheme that could be used for re-distribution of original footage and to be able to quickly pick and pluck from the archive at will. If there is a really good way to do that using the capture codec(s), I'm all ears.

You certainly can archive your recording codec to a hard drive. Since you must have started with DVCProHD (5 years ago), and now use Intra, and if you stay Panasonic, will likely migrate to some form of Ultra, and maybe will use any number of other recording formats in the future, does this cause you any difficulty when you want to view some archived footage for review and be able to quickly pluck some frames out of a clip? Does archiving in the recording codec serve the purpose of the OP? Having the metadata in excel is great, but it isn't hotlinked to the footage is it?

I think this is a question of whether you want to do more organizational work up front on your archives or prefer to do it later. Is there a right answer here?

Grant
 
ggrantly said:
My point was to archive in a consistent format which is universal and ProRes is certainly that, though if you are totally PC based, another codec without the need for encoding would be better.
Click to expand...

ProRes is also Apple proprietary technology, and if they never open source it like they did their lossless audio codec, they could choose to drop it at any time. Seems like a risk to me.
 
No doubt in the fast changing world of codecs, whatever you hitch up to could evaporate. Is there anything that would be a better bet. I mostly live in the Apple realm, so I'm not so familiar with the alternatives or how well they work. I do know that Avid-ProTools-MAudio is bleeding. Grass Valley, Sony, Adobe? They aren't the creators of codecs are they?

Grant
 
I think the, by far, safest option is to leave everything exactly as it came out of the camera. That is probably the best bet.

I know in your original post you mention P2 and wanting to trim the fluff. I didn't read back through the thread to figure out what you'd been told already, but I do this all the time (adding/removing footage, combining cards into one P2 structured folder, ect) using P2 Viewer on Windows 7 x64 and P2CMS on OSX Lion (though it sucks on Lion because it will no longer playback the footage).
 
Seems like a reasonable compromise (all of which has already been discussed here) might be to select your clips from your NLE timeline, then send them back out to P2 (I know Edius Broadcast does this). Then tag your new P2 files . . . Excel or a text file.
Then achieve it to, as Jan suggested, LTO tape.

While I don't think there exists (yet) a perfect solution, this hits a few high points: original format, straight forward work flow, probably the least amount of man hours (with less pulling out of hair) so time/money are contained at a very reasonable cost.

I've been achieving to LTO since my first HVX-200 (and that was a while back), without complaint.
 
Ok, one step closer for us in the hunt for a user-friendly system to build a easy to navigate library, but a bit of testing of some new equipment was required..
So, the call was made to go for a shootout between our 250's and the new PMW200 with the expectations of jumping ship to XDcam. The PMW200 looked and felt amazing, with its size and solid built feel to it. The Fujinon lens and the rings had a very nice feel to them and there were many good things ("initially") it pretty much has the similar buttons, in similar positions as the HPX250, but only 2 ND filter settings, which was an off-turner for us. Autofocus: You would expect the PMW200 to be superior when thinking of all the users complaining about the HPX250 (which has now been fixed) but in my opinion they were pretty much identical. Low light, the PMW200 had a slight advantage, but not as much as we expected, in fact: Very little....?? But worst off all, and our reason why we walked out the shop with yet another brand new HPX250 was the LCD and the Viewfinder on the PMW200. The LCD is 'ok' but nowhere near as good as on the HPX250. I could possibly live with that, but when testing the Viewfinder???? " i think unusable is the best description.....
Still the XDcam codec would make it so much easier for us to build an easy to use, easy to build, footage library, but we simply could not shoot with the PMW200, so whats the point.... So back to the drawing board and the search of finding a helpful software we go again..
Has anyone heard of, or used CatDV media management from http://www.squarebox.co.uk?
Thanks!
 
Last edited:
Oh thats interesting. I was sure the 1/2" sensors of the PMW 200 were a winner over the HPX250´s 1/3". A lot of people in my circles are exited about this new Sony and dont like the HPX. Regarding your media management, @ 300 a pop that solution sounds worth a try. Never heard of it though. I would simply back up the P2 folders separately anyway. Rule is 3x, so get a lot of drives and service them regularly since the grease can get dry.
 
You must log in or register to reply here.
Back
Top