GH2 AVCHD - Why so many files?

G

guyburns

Active member
This AVCHD thing has got me beat. I thought I remembered reading in a post several years ago that you had to import the entire PRIVATE folder to get things to work. However, if I plug my SD card into the side of my iMac, go to PRIVATE > AVCHD > BDMV > STREAM, copy the 00000.MTS file to the Desktop and remove the SD card, 00000.MTS (44.5 MB) appears to play perfectly in VLC.

If I then use ClipWrap to rewrap the .MTS to .MOV format for playing in Quicktime, it also seems to play perfectly (with a slight increase in size to 46.5 MB).

Now, the PRIVATE folder which contains the clips has a size of 5,464,674,734 bytes. The size of all the clips inside the STREAM folder is 5,460,510,724 bytes. Thus there are only 4,164,010 bytes of overhead, i.e. 99.92 percent of the data is held in the MTS files.

At the moment the PRIVATE folder contains 338 files, 116 of which are, to me, useful MTS files, and they seem to work okay all by themselves without need of the remaining 222 files. I don't understand why the extra 222 files are there. I don't need to know the complete ins and outs, but for archive purposes why can't I just ditch everything but the MTS files?


SUMMARY
Below is a summary of responses, mostly for my own benefit. I also treat these things as a learning exercise, so I've researched all the obscure terms I didn't know much about (such as MTS) and explained them here as well, again mostly as a summary for me. Maybe someone else will find it useful.

Note, ClipWrap ($50) is mentioned here a few times. There is an alternative free converter: http://media-converter.sourceforge.net. For comments about the two, see https://discussions.apple.com/thread/3131540?start=135&tstart=0


1. When recording in AVCHD mode on a GH1 and GH2, the video and audio content is stored inside the PRIVATE > AVCHD > BDMV > STREAM folder with the suffix .MTS.

2. MTS is a variation of M2TS (MPEG-2 Transport Stream), the container file format used for multiplexing audio, video and other streams on Blu-ray disks.

3. In addition to being a Blu-ray standard, MTS is used in the AVCHD format but with restrictions. Only one video compression format (H.264/MPEG-4) and two audio formats (Dolby Digital: AC-3, and LPCM) are provided. For the GH1/GH2 cameras, audio is encoded only in the AC-3 format at 192kbps (48kHz sample rate, 16-bit depth, compressed).

4. As well as generating an MTS file of the recorded video and audio, the camera generates other files inside the PRIVATE folder associated with the movie clip. None of these extra files are required to play or archive the video and audio content of a clip, except in one case: where a clip exceeds 4GB (a limit imposed by the FAT32 file structure used for the SD cards). If a clip exceeds 4GB, the camera creates a second MTS file that is linked to the first file. The two (or more) linked MTS files are said to be "spanned".

5. Although the additional files are not necessary to play or archive a movie clip (unless spanned), they may contain useful information such as timecodes. The GH1/GH2 do not offer timecodes. Gary Huff explains it this way (http://www.dvxuser.com/V6/showthread...post1986246980). I have slightly edited the following, and other quotes herein:

The general desire to keep the PRIVATE folder intact is because certain higher-end cameras will record miscellaneous data in the structure. Your GH2 will not (the GH3, on the other hand, with its timecode abilities is a different story). There's no metadata or anything to worry about losing, so feel free to move the files around to your heart's content… With GH2 footage, nothing bad (outside of spanned clips, and even then you lose a frame or two) will happen if you do that.​

6. MTS clips (unless spanned) can be renamed and exist as playable, archivable, standalone movie files.

7. MTS clips can be made more widely playable by rewrapping them with ClipWrap to the MOV format. The video is unaltered; the audio is converted from 16-bit to 24-bit, thus the file size will increase slightly.

8. For spanned MTS clips, ClipWrap can seamlessly join them as part of the rewrapping process.

9. Standalone MTS clips can be imported into Premiere CS6 and Final Cut Pro X (but not earlier versions of FCP).

10. Standalone MTS clips can be imported by Pinnacle Studio, PowerDirector, Edius Neo 3 and Vegas.

From Bruce Foreman (http://www.dvxuser.com/V6/showthread...st1986246981):

Pinnacle Studio, PowerDirector, and Edius Neo 3 work just fine with just the MTS files. If your computer has enough CPU horsepower and you're using an NLE that does not need all that extra stuff, like the three I've listed above, the MTS files can be dropped right on the timeline with no transcoding and be edited straightaway… I've been editing AVCHD ever since Canon put the HF100 on the market and I've never had to diddle with anything but the MTS files. You can keep it simple.


From Shenan (http://www.dvxuser.com/V6/showthread...st1986247281):

Vegas handles pure MTS files without the folder structure beautifully. The only limitation that I've run into so far was on interviews where the clip went over the 4GB file limit…​

11. From Pete (http://www.dvxuser.com/V6/showthread...st1986247273):

You have to be careful when deleting or cutting MTS files. Once the MTS file is gone it is impossible to delete the other directories and files from within the recording device. If you reformat your cards on a regular basis you won't have to worry about those files building up and robbing card space.​
 
Last edited:
Question for you, how long are the clips that you shoot? You might end up finding a glitch in spanned clips using your method. I don't know the reason for the file structure but I have read several threads about people having issues with clips spanned between multiple .mts files when they are removed from the file structure. I've been using the GH2 for over a year and a half, mostly using final cut pro 7 and transferring everything to prores, Log and capture in fcp7 will not recognize .mts files taken out of the structure. Now I'm mainly using Premiere Pro CS6 and the structure is also necessary to use the media browser correctly. Then I edit the AVCHD natively saving a ton of drive space.
So yes there are a lof of additional files in the folders but the format is so efficient that the space they take is almost nothing. You might have already known everything I just wrote but I think copying the private directory is the best way to go if you are just shooting video.
 
I think the best way of archiving cards, at least for a mac is to just make a disk image using Disk Utility.
 
guyburns said:
i.e. 99.92 percent of the data is held in the MTS files.
Click to expand...
Yes. Which means that all the other stuff, all added together, takes up only 0.08% of the space. So what's the resistance to keeping it?

I don't understand why the extra 222 files are there. I don't need to know the complete ins and outs, but for archive purposes why can't I just ditch everything but the MTS files?
Click to expand...
The system thought they were important. It doesn't just go arbitrarily creating random files, it makes those files on purpose because it thinks they're useful and it may use them. If you have them, you're covered. If you don't have them, you may be okay, or you may be pooched. Is it worth the risk, to not have them, to save 0.08% of disc space?

At the bare minimum, the additional files tell AVCHD-aware devices (such as blu-ray players, xboxes, playstations, and televisions) how to play the footage, what the clip names are, which clips are tacked on to other clips, and -- if you have a camera that's capable of it, those files would also include metadata such as timecode or information about the clips that you could enter and track.

Assuming you're using a GH1 or GH2, when are you likely going to need those files? Usually only if you shoot footage that "spans" a clip across two or more files. The metadata tells the NLE how to properly stitch the clips back together into one big long seamless clip, with no gaps inbetween. Without the metadata, you'll probably someday encounter a scenario where you have a spanned clip, and when you put the two pieces on the timeline, you'll find there's a gap of about 1/2 second where they meet. The metadata fixes that.

Another place where you'd need that data is if you're using an NLE that doesn't recognize the MTS files. FCP, for example, requires the entire folder/directory structure and all the files in them to be present, or it doesn't know how to import footage. FCP users would be utterly screwed if they didn't have that info -- they would be relegated to absolutely having to use a tool like ClipWrap, because FCP itself just wouldn't work with the MTS files at all.

So -- in short, you are saving effectively zero by ditching the files, but you're potentially cutting yourself off from options that you would have, if you kept them. So my advice is to always keep them.
 
What Barry said....
Also, I don't understand the big cloud of negativity towards the AVCHD format? I get it from canon dslr shooters all the time... What is so hard about copying the whole AVCHD folder as opposed to just the .MTS files to your HDD so your NLE can see them, and stitch them back together if spanned? It's literally NOT an issue at all. I don't understand why people hate it so much.
 
I think it's because people are jumping into this digital workflow without having a fundamental grasp on the basic concepts of stuff like dealing with file/folder structures. I'm sure if you took a look at how their projects were organized, it'd make more sense.

Here's my workflow: copy entire contents of card to "Footage" folder underneath main project folder. Navigate Premiere CS6 Media Browser to that folder. Import. Done.

No muss, no fuss.
 
Gary Huff said:
I think it's because people are jumping into this digital workflow without having a fundamental grasp on the basic concepts of stuff like dealing with file/folder structures. I'm sure if you took a look at how their projects were organized, it'd make more sense.

Here's my workflow: copy entire contents of card to "Footage" folder underneath main project folder. Navigate Premiere CS6 Media Browser to that folder. Import. Done.

No muss, no fuss.
Click to expand...

Mine is similar... I also create a "Footage" folder in the main project folder, but in that folder because I use multiple cameras and cards, I separate them by operator name. So: Project name\Footage\Operator\Card_1\AVCHD\ (and Card_2). Then in Premiere Bin I just drag each Operator folders into the bin and it'll create bins with operators names/Card_1 and then all the .MTS files are seen. Very easy and organized.
 
Yep, also similar to mine. Mostly there's just one operator (me) and one card, but in the event of multiple cameras/cards, I do the same thing.
 
ATL Media Group said:
What Barry said....
Also, I don't understand the big cloud of negativity towards the AVCHD format? I get it from canon dslr shooters all the time... What is so hard about copying the whole AVCHD folder as opposed to just the .MTS files to your HDD so your NLE can see them, and stitch them back together if spanned? It's literally NOT an issue at all. I don't understand why people hate it so much.
Click to expand...

I usually don't clear off my card after my typical low end jobs that I do every week. Of course, for bigger projects that require more planning and organization, I have to clear them off since the cards will be full after one or two days of shooting. So for the run of the mill stuff, if I was copying the folder each time, I would be putting the same videos on my harddrive more than once and wasting space. But I'm still using FCP7 and transcoding each file, so this has never been a problem in the past. I want to move to Premiere CS6, so I'll need to learn the proper workflow when I do.
 
ATL Media Group said:
I don't understand why people hate it so much.
Click to expand...

Because you can't do any kind of file management on the actual video files and still have everything work. We can't move or rename the clips like we could with DV captures. For best functionality, the folder structure needs to remain intact and unaltered, and sometimes that sucks!
 
Jim Simon said:
Because you can't do any kind of file management on the actual video files and still have everything work. We can't move or rename the clips like we could with DV captures. For best functionality, the folder structure needs to remain intact and unaltered, and sometimes that sucks!
Click to expand...

I always labeled the folders instead of the files but I could see where it would be a PITA if you always renamed the files themselves.
 
Thanks for all the responses. I'm new to video, so bear with me. I must stress that I am not trying to be difficult. I am a complete newbie to hands-on video who is wanting to find out exactly the way things stand – for my purposes. I just don't want to be blinded by convention. I'm the child in front of the MTS Emperor. I can see him bare, whereas others see the multi-layers of folders surrounding him.

Below are my responses to comments from posters, and some of my own comments and further questions. But first a summary.

My clips will be shot on either a GH2 or GH3, 24p, 1920x1080, highest quality setting, and will never be spanned. Never. Most clips will be less than a minute, rarely above 2 minutes. I may capture thousands of sound and video clips, contained within hundreds of PRIVATE folders, taken over a time frame of five years.

Assume in what follows that when I say "clip" or "file", I mean a clip as described above.


Can a standalone MTS clip be imported into FCP/Premiere?
I'll probably be using Premiere to edit. Is it definitely the case that Premiere and FCP cannot, under any circumstances, make use of a standalone MTS clip? If yes, I don't understand. VLC can play it,. my Oppo BDP-93 can play it. All the info appears to be in the MTS file. Does Premiere lack the appropriate software to decode what is inside an MTS file? Can someone with FCP or Premiere confirm that a standalone, unspanned MTS clip is unusable?


Can an MKV clip be imported into FCP/Premiere?
Can a standalone MTS clip, rewrapped to MKV by ClipWrap, be used by Premiere/FCP?


Quote from Barry
At the bare minimum, the additional files tell AVCHD-aware devices (such as blu-ray players, xboxes, playstations, and televisions) how to play the footage…

Well, some of my stuff knows how to play the MTS file without the other files. I have just copied an MTS file, renamed it, saved it to a bare USB stick, plugged it into my Oppo Blu-ray player and it played perfectly. Did the same with VLC on my iMac. And, after rewrapping with ClipWrap, it played in Quicktime.

But, maybe I only think my standalone MTS files are playing perfectly. Maybe something is missing that I am not detecting. Is that possible?


Another Quote
Assuming you're using a GH1 or GH2, when are you likely going to need those files? Usually only if you shoot footage that "spans" a clip across two or more files.

Since I won't be generating spanned files, this does not affect me.


And Another
So what's the resistance to keeping it [the PRIVATE folder]?
If files other than MTS inside the PRIVATE folder are absolutely necessary, fine. But if not, I don't want them, or the structure they inhabit, no matter how little space they take up. I'm big on efficiency. I imagine that if I'm putting together a HD audio-visual from several thousand separate elements (stills, video clips, sound clips, titles), that I'd like to name the individual elements with appropriate names ("Jenny Walking 001", "Greg Eldon Commentary 017", and so on), put them all inside one archive folder and a variety of subfolders, and link to them.

Maybe I'm wrong, but I'm guessing that the general technique for assembling 1000 sound clips and 2000 video clips into a 60-minute audiovisual, is pretty much the same as the general technique of assembling 1000 linked images and 2000 text frames into an 800-page book such as Tanksinker (check out https://sites.google.com/site/tanksinker/Home/max-burns-tanksinker). If I can get away with standalone, individually-named MTS clips, indexed in a database, stored inside a single folder and linked to an audio-visual project – that's what I want.

My biggest problem with PRIVATE folders (correct me if I'm wrong) is that I may end up with hundreds of them. My projects (transferring 3-screen AVs, about 20, to HD format, and generating new video shorts) may extend over more than 5 years. How do I distinguish the many different PRIVATE folders I'll end up with on my computer?

Here's what I mean. I've got a 3-screen AV from 1985 called Clouds. Apart from the 300 slides it contains, I'm going to add video clips over the next few years as I capture interesting cloud formations. In addition, I'll be videoing for other purposes, with all the clips (could be 10,000 of them) nicely intermingled amongst a few hundred PRIVATE folders. What a mess that would be in terms of indexing and locating individual clips. My experience with Tanksinker is that indexing and appropriate folder design is imperative from the outset. I reckon that 5% of the eight years I devoted to Tanksinker was spent on indexing and archiving (of the 56,000 files that made up the final 58 GB archive, I never lost one in eight years).


Am I on the wrong track for video?
I may be completely on the wrong track. In which case, please let me know. But compare the mess described above with grabbing each video and sound file as it comes off the SD cards, throwing away the duds, renaming the good ones, indexing them, saving them to appropriately-named folders and sub-folders, and then, years in the future, finding them again and linking to them from Premiere. For example, if in 2016 I want to locate that Cumulus thunderhead that I vaguely remember videoing in 2013, I'll go to my index, search for Cumulus, and up will come 45 different clips, indexed by location, year, clip-length, camera used, overall description, and where the clip is stored. In less than 30 seconds (a typical time for me to find any of my Tanksinker files), I'll find the Cumulus clip I want, and import it.

Retaining PRIVATE folders instead, all named PRIVATE, containing clips with descriptive names such as 059817.mts? I can do without them – I hope.

This video child is seeing the MTS Emperor as bare. Does anyone else?


Further background
I'm approaching this video stuff very cautiously. I'm not sure it's my thing. What definitely is my thing is to convert my 3-screen audio-visuals to HD format, and I'm toying with the idea of adding video clips here and there. Thus my interest in the GH2 and GH3.

I'm also setting up a home theatre, 1950s style, for showing my AVs and movie classics. Imagine canvas seats, popcorn, and a "cigarette" girl (my partner) with her tray of ice cream, lollies and other treats (no cigarettes nowadays, of course). The first half will feature a cartoon, a news reel, maybe a Three Stooges episode, one of my AVs, and a GH2 short. Then an interval, when the cigarette girl comes into her own, followed by the main feature: some classic, pre-1960s movie on Blu-ray. Each half will be locked into an MKV file (20-40 GB). Thus my interest in MKV.

At the appropriate time I press a button on the Oppo player feeding into a BenQ W7000 projector and up comes the image on a 135" screen. Using the latest technology to transport an audience back 60 years. Yay! Looking forward to it – but without all those PRIVATE folders.
 
Last edited:
guyburns said:
Can a standalone MTS clip be imported into FCP/Premiere?
Click to expand...

Yes.

Can an MKV clip be imported into FCP/Premiere?
Click to expand...

No and yes. For Final Cut Pro Legacy (either version 6 or 7), you should convert the .MTS file into a ProRes 422 .MOV file, either using 5DtoRGB or Magic Bullet Grinder or MPEG Streamclip. For Final Cut Pro X, you can use it standalone.

Maybe something is missing that I am not detecting. Is that possible?
Click to expand...

If it works for you, why sweat it?

Here's the deal: the general desire to keep the PRIVATE folder intact is because certain higher-end AVCCAM cameras will record miscellaneous data in the structure. Your GH2 will not (GH3, on the other hand, with its timecode abilities is a different story). There's no metadata or anything to worry about losing, so feel free to move the files around to your heart's content.

I think at the very least, being on the Mac platform, that you invest in ClipWrap (available from the App Store) and re-wrap your .MTS files to .MOV, simply for ease of use with playing them back. Until Mountain Lion (kind of a ***** to deal with AVCHD in Mountain Lion because of the way it processes the folders for you) it was impossible to get decent playback from just bringing up the file. If you use ClipWrap, then you can use QuickTime and you'll be easily able to call up clips to view them outside of an NLE. Otherwise, just organize in a way that makes you happy. With the GH2 footage, nothing bad (outside of spanned clips, and even then you lose a frame or two) will happen if you do that.
 
Guy,

I don't know about Premiere, I've never used it. But Pinnacle Studio, PowerDirector, and Edius Neo 3 work just fine with just the MTS files. If your computer has enough CPU "horsepower" and you're using an NLE that does not need all that extra stuff, like the 3 I've listed above, the MTS files can be dropped right on the timeline with no transcoding and be edited straightaway.

Typically I copy the MTS files straight from the PRIVATE>BDMV>STREAM subfolder to a temporary project folder on an external hard drive (drag 'n drop) where I can review them and do a quick rename to a project name with "shoot" name (like "FtChad-CavDrills-0001") that will mean something to me in the editing process. Then the renamed files are copied to a project folder on each of 2 external drives. Makes editing much simpler.

Storage is even simpler. I can keep those project folders with the renamed MTS files on those external drives, copying what I need to the work drive. Pinnacle Studio 15 has a neat archive function that saves the timeline and all media files in one archive folder, and it looks like version 16 (which is really what would have been Avid Studio 2 if Avid hadn't sold their consumer stuff to Corel) has the same thing. I have it but haven't installed it yet.

I've been editing AVCHD ever since Canon put the HF100 on the market and I've never had to "diddle" with anything but the MTS files. You can keep it simple.
 
Gary Huff said:
Only if you have spanned clips.
Click to expand...

And that's the other disadvantage of AVCHD - spanning. With DV, a one hour take came out as one file. You could do anything you wanted with it on the hard drive and nothing got broke.

That same user friendliness should have been the model for solid state media. This whole FAT32/folder structure thing is just plain wrong.

One file to hold video, audio, timecode and other metadata, recorded onto NTFS or exFAT formatted media.
 
This is the file structure for AVCHD. It is exclusively for playback on dedicated devices like blu-ray players. All useful information for editing and playback are contained in the *.mts file.

380px-AVCHD_actual_file_structure_svg.jpg

A new AVCHD hierarchy can be generated at a later date for most any file with http://multiavchd.deanbg.com/.

One warning, you have to be careful when deleting or cutting *.mts files. Once the mts file is gone it is impossible to delete the other directories and files from within the recording device. If you reformat your cards on a regular basis you won't have to worry about those files building up and robbing card space.

Cheers,
Pete
 
You must log in or register to reply here.
Back
Top