Audio and frame rates?

[Noiz2]

While I agree with your point which is always work at the actual frame rate, the below is not necessarily true. Frame rates have nothing to do with audio speed, unless you are clocking at a different tate. So in ProTools you can have a problem depending on how you output.

In your case I suspect the problem was more on the other end but? If you output a mixed file with a frame rate (in the metadata) that doesn't match the "capture settings" in FCP it will sample rate convert on the fly with no notice to the user. That will throw you out of sync. ProTools though is clocking everything the same so you can just switch your frame rate to match and that same file should work. Or the more usual fix is to wrap the sound as a QuickTime File with the frame rate that FCP has set in it's capture settings and then FCP wont "convert" your SR on import.

Point is the sync in your DAW, well ProTools anyway, is correct. It is frame to frame in sync and IF you have been ignoring the TC in red on the video track (that BTW is the indicator that your session FPS doesn't match what the video FPS metadata says it should be) then you can correct that at any point and your session is still in sync.

Just so you know, if you cut picture in one frame rate, do sound editorial in another, and then give final sound to picture to put them together, there will be drift. Happened to me on the very first short film I ever worked on. I was told pictures was a 30 when it was really at 29.97. I worked at 30. Drift ensued when we handed off the final mix. ALWAYS make sure your frame rates are discussed and understood. 23.976 does NOT equal 24, and 24 does not equal 23.976. I've had so many camera people say "yeah, I'm at 24", when they really mean 23.976, so when I hear that I always ask "do you mean 24, or do you really mean 23.976?", to which I usually get the answer of "uuuhhhh...I don't know. How do I find that out?". That's happened to me everywhere from MOCAP stages, to low budget stuff, to $200 million union features....

 

[Noiz2]

This is correct -ish IF the sound designer was playing back 23.976 video at 29.97 FPS. IF though the SD was given a 29.97 output to work at then you should be fine because the "speeds" are the same. If the designer for some reason has changed the speed of the video on his end he should A) be tied up and whiped and B) fix it because he did something very naughty.

IF the SD is long gone then you can probably wrap the output file in a Quick Time wrapper with the projects FPS and get it to come in in sync.

The way to find out is to see if the two files are the same length (before bringing the sound into FCP) so you know if there is an actual length issue of if it's a FCP import issue.

A correction to the below is that it's not the 3/2 part that trows off the sync it's the 30FPS played at 29.97 FPS when that 3/2 pull down from 24 > 30 video frames gets played back.

In the OP's case both frame rates are at video speed so properly done they would play in sync. Which means, since they don't, that some where the speed changed. Either the SD improperly converted the 23.976 or the NLE is doing a frame rate conversion on import.

If wrapping the file doesn't fix it then pull it into a DAW and time compress it by 0.1% and you should be back in sync.

In order for the 23.976 material to be played correctly at 29.97, the material needs to be converted to 3-2 pullup sequence. What this does is take 4 23.976 frames, and spread them across 5 29.97 frames. (Which has been done since NTSC began and 24 fps movies converted for broadcast...).

Anyway, at 23.976, there are 2002 audio samples per frame @ 48K/s, and for 4 frames that would be 8008 samples.

For 29.97, there are 1601.6 samples per frame @48K/s, and after 5 frames, one has 8008 samples.

In other words, by the time all is said and done in the 3-2 conversion the number of audio samples across 4 frames at 23.976 equals the same number of samples for 5 frames of 29.97.

And all is well in the world...

On running 23.976 material at 24 fps without compensation, the reasoning is thus.

There are 2002 samples @ 23.976 from the above calculations.
There are 2000 samples @ 24 fps.

So after 1 second of play @ 24 fps, the audio is 'short' 48 samples. After 10 seconds, it is short 480, after 20 seconds it's short 960... which is almost 1/2 a frame, and so can impact lip sync, or other audio/visual cooccurances.

 

[Noiz2]

@Noiz2 Thank you for your very informative answer. Looking at that 0.1% I suspected where it all came from; however, I still had an uneasy feeling that there were a chance I could do something wrong. Now I know that with I do, I can go ahead with 48kHz!

I can think of virtually no scenario today that would want to use a non standard sample rate. NOBODY is working on video tape which was really the only reason they existed.

The simple answer is "are you shooting on 35mm film?
If the answer is no then you can forget the odd rates.

If the answer is yes ask the producer or post coordinator (Ideally it would be the sound post supervisor but good luck with that) what they want the production sample rate to be. Today in 99.999% of the cases they will want it at 48K.
If they don't know then they want it at 48K.
If they are confused by the question they want it at 48K.
In any case where someone has not put it in writing that they want it at something other than 48K, they want it at 48K.

Think of the odd rates as a nod to a window in history.

 

[Noiz2]

Then tell them to get Pro Tools HD. It can handle any standard frame rate. I've never ever heard of someone only being able to work in 29.97 that was actually doing real work. Honestly, if I were in this situation, I'd hand off what I recorded and let them know they need to convert it to 29.97 since the sound designer can only work with that frame rate. It may seem jerk-ish, but having been down similar roads like this before, here's why.

If I say that I'll do the conversion, now I'm responsible for making sure it works properly wherever it is going, and if I don't feel confident in the person that I'm delivering the files to handle whatever I give them, no matter what it is, then I'm going to get thrown under the bus no matter what. So, If you rolled at 23.976 on production, then that's what you hand off. You tell them that and then collect your paycheck. There are post production tools to help with post production that they can use. Now, if the post guys were capable and nice and didn't get their jobs because of nepotism, then I'd be happy to help with whatever needs to get done. I've been burned too many times by amateurs, so once bitten twice shy. I won't be a complete jerk. I'll point people in the direction of the stuff that'll help them, but that's about it.

Well... I can think of some reasons, though no really good ones. If you are monitoring on an old TV then 23.976 is not going to play correctly. And some people (Even hot professionals) have old systems they are in love with that have certain "limitations".

OP,

IF YOU are doing a 29.97 output for him from a 23.976 project there should be no actual time differences between you, they are running the same real time speed.
If you then have an issue on import it's probably because the NLE is one of these new "easy" "it just works" editors that pretty much take a bunch of important decisions out of the editors control and have programers hard wire choices that THEY think are the right ones.

SOOOO, you have to trick the software, or compensate for it's weakness. You can trick it by changing metadata on the file so it now says it is at 23.976 FPS. Sebsky tools will do this http://www.askincstudios.com/zz_misc_files/stuff//SebskyTools.zip It's an older Mac. program so it may not work anymore. Wave Agent might do it also, haven't tried my self yet.

The other option is to figure out what mean thing the NLE is doing and time shift the file in advance so that when the NLE messes with it it just undoes what you did.

Because of your description of the problem though I suspect the actual issue is not 23.976 VS 29.97. I suspect that you are on FCP and the problem is one of two things.
The first is that IF the Capture settings in FCP do not match the project settings it will Fux up your sound on import. This can be corrected by either of the options above.
OR QT is incorrectly saying the sound file is at 30 fps. QT is often wrong when it looks at frame rates, but this is a lot less likely the problem than the above and the fix is the same.

The shrinkage BTW if either of the above is the case will be 0.1%.

 

[Noiz2]

NTSC drop-frame has been a nightmare since digital started to come into play in the early 90's

THe only time drop frame is an issue with sync is if you are locking two devices together with timecode as you timing reference. Since only analog gear can sync to such a crude clock this has been a non issue for a LONG time. And DF came in with color TV, so 70's.

Though it can be a bit of a pain with TC not matching so you cant go to TC X on two different machines easily. And you could I guess get a hiccup doing a digital tape to tape because even though your clock was separate the TC might throw an error when the drop happens.? From practical experience the only real two problems I have ever seen with DF were the PITA of converting notes from DF to non DF when someone took notes in the wrong mode and the odd case where someone who didn't know what they were doing decided a show should be in drop frame when it shouldn't and you had an odd person at a wrong rate. But the later is ONLY an issue if all you have is a tape. In the session you just switch a setting since there is NO speed difference between drop and non-drop.

What you may be confusing and IS a nightmare is the move to 29.97 FPS. DF was invented for use with 29.97 so that the TC would not be way off compared to "real time". However other than broadcast where timing is critical nobody used DF. In a film 3 seconds an hour are insignificant (remember this is not a frame rate change just a different way of counting those frames).

But you should never have had to deal with DF as it was only really used in US broadcast shows.

 

[Noiz2]

Sorry for the blast of posts, I've been away...

 

[davefisk]

While I agree with your point which is always work at the actual frame rate, the below is not necessarily true. Frame rates have nothing to do with audio speed, unless you are clocking at a different tate. So in ProTools you can have a problem depending on how you output.

All I can tell you, based on experience with several projects, is when I was told picture was working at one framerate, and I was working at another, that picture and sound ended up drifting when put back together. It's happened with Avid and FCP. Maybe there was something else going on in the picture end. All I know is that frame rates weren't the same, and when things like that aren't the same, problems occur. I've seen it happen a billion times when I was a product specialist at Coffey Sound. If I had a dime for every time camera said they were at 24 and they were at 23.976, and sound rolled at 24, and post calls about drift, and I have to go solve it...it was always a frame rate issue...if I had a dime for every one of those, I'd own my own island in the south pacific.

 

[GaryNattrass]

I perfectly understand that drop frame timecode ala NTSC colour is not really relevant for HD but people still seem to be quoting frame rates and more importantly drop and pull up sample rates.

Most modern digital kit should be gen locked for video work and timecode is just there to give a positional reference but you will get an edit looking at 30 frames for convenience but then running drop sample rates.

Like David we also had huge problems at AMS Neve when I worked there in the early 90's and you would get the edit, the camera, the sound recordist and the Dub all potentially doing different things.

Then you get people wanting to shoot cinema film rates for TV and vice versa and then you end up with the dolly mixture of sample and frame rates.

As I said here in the Uk it is 1080 50i or 25p for TV and 24p for cinema but then TV will just play back a 24p film at 25fps. Sample rate tends to always be 48k 24 bit.

 

[Noiz2]

All I can tell you, based on experience with several projects, is when I was told picture was working at one framerate, and I was working at another, that picture and sound ended up drifting when put back together. It's happened with Avid and FCP. Maybe there was something else going on in the picture end. All I know is that frame rates weren't the same, and when things like that aren't the same, problems occur. I've seen it happen a billion times when I was a product specialist at Coffey Sound. If I had a dime for every time camera said they were at 24 and they were at 23.976, and sound rolled at 24, and post calls about drift, and I have to go solve it...it was always a frame rate issue...if I had a dime for every one of those, I'd own my own island in the south pacific.

I agree that you should be at the same frame rate. However ProTools clocks everything the same. So if you have a 25 FPS video and your session is set at 24 fps, the TC on the video track turns red (far left in track header) so you know you are in the wrong FPS. IF you switch back then you are in the right FPS and everything is fine. Everything you cut when you were in the wrong FPS will slide to where it should be and YOUR session will be in sync and at the right FPS. You are always working frame to frame accurate.

But indeed you do have to pay attention. And I completely agree that picture on low budget projects especially, often have their head where it doesn't get sunshine and are OFTEN referred incorrectly to their real frame rate. THe only slack I give them on that is some consumer gear incorrectly labeled their FPS so the camera people have to actually know rater than just going by what is in the menu. This is especially true with 23.976.

The place where we split is how total melt down the situation is. Sound (professionally anyway) is almost constantly having to fix things that should have been done correctly on the picture side but were not, so I have been down this road before in many variations. I have a LOT of utilities that I got purley to be able to alter things that come from or go to picture to match some weird issue they have, generally because someone didn't know what they were doing.

So while ProTools can fix it's compatibility issues in many cases FCP once you set up the project with mismatched (Project/Capture) settings and put anything in the timeline is hosed. At least I haven't found anyone who has figured out how to fix the project. So once that happens you will never be able to send sound that syncs to picture and have it come in in sync. You have to lie to FCP to get it to leave the sound alone.

 

[GaryNattrass]

I think in 25 years of pro audio post I can totally agree that it is always the audio dept that has to fix all the crap that comes from the picture department and production people who at times don't even know what format to shoot on!

 

[Noiz2]

I perfectly understand that drop frame timecode ala NTSC colour is not really relevant for HD but people still seem to be quoting frame rates and more importantly drop and pull up sample rates.

Most modern digital kit should be gen locked for video work and timecode is just there to give a positional reference but you will get an edit looking at 30 frames for convenience but then running drop sample rates.

Like David we also had huge problems at AMS Neve when I worked there in the early 90's and you would get the edit, the camera, the sound recordist and the Dub all potentially doing different things.

Then you get people wanting to shoot cinema film rates for TV and vice versa and then you end up with the dolly mixture of sample and frame rates.

As I said here in the Uk it is 1080 50i or 25p for TV and 24p for cinema but then TV will just play back a 24p film at 25fps. Sample rate tends to always be 48k 24 bit.

??? HD??? In the 70's on VHS? Don't think so.
Drop frame is ONLY a change in how the TC is counted and has NOTHING to do with speed, pull up etc.

You guys got it right! We had a chance at getting it right but political pressure and a lot of $$$$$ forced us to do it wrong.

But what was / is wrong is the STUPID fractional frame rate. Drop frame simply "drops" a frame periodically to compensate for the fact that the idiots in the US are playing 30FPS slowed down to 29.97 FPS so if you count normally you end up with TC that doesn't match "Clock Time". You can't have a fractional frame so either you count as normal and end up going farther and farther away from clock time or you skip over a frame here and there so that you stay reasonably close to "clock time".

It is exactly like Leap Year. The year goes by at the same speed but we jump over a day so that the calendar doesn't get too far out of sync with the sun.

There is no such thing as a "dropped sample rate". Trust me on this and you have a right to be confused. But I grew up with this steaming pile of TC. You can get into deep trouble confusing TC "counting" with actual speed changes, at lest in the US.

With any luck this will disappear in my life time and people can talk about the dark days of NTSC. But till that happens one needs to keep how you count frames and how fast those frames go by in separate cubby holes. With actual film a frame is a frame and you actually work at the 1/4 frame (at least in sound) because of sprocket holes. Because of the nature of video there are only full frames, but TC is not necessarily tied to the frame. LTC not at all, VITC is, and files are not in the least. I can have a 25 FPS video with 30DF TC, it doesn't really mater AS LONG as any tape or box I want to sync is ALSO running 30DF TC.

But as we both have kind of been saying NOBODY is syncing tape these days so nobody is using TC for a sync reference. File based stuff only use TC (if at all) for positional reference. If you have mismatched TC you won't drift out of sync as you play but if you stop and jump in to point X you wont start in sync.

Mix stages maybe one of the few places left where you might have to deal with TC drift because of a mismatch. And in the US that would be VERY few mix stages. With digital everything is getting clocked unrelated to TC so speed is very consistent. You could have positional issues but then you are just going to change you=r TC settings to match. With TAPE this was a very big deal and you had few options if the TC was screwed up, not really the case today.

 

[GaryNattrass]

??? HD??? In the 70's on VHS? Don't think so.

????

Did I mention VHS the 70's or HD relating to that time in my post ???

Sample rate is 44.1 or 48k in my book of digital so if you are clocking 44.056 or 47.97 that is to suit drop frame.

If it is irrelevant these days in the USA then why are we even discussing it.

Surely HD in TV land is 1080i or 1080P depending if you want a video or film progressive look?

Read Alan Roberts book as he is my reference point but sadly too many semi pro's are still doing stoopid things!

P.S I havent even mentioned field order and you should see some of the horror that comes thru the BBC these days with all sorts of muppets shooting with their DSLR's and go pros set at all sorts of frame rates and codecs. :grin:

 

[Noiz2]

??? HD??? In the 70's on VHS? Don't think so.

????

Did I mention VHS the 70's or HD relating to that time in my post ???

No actually you said it wasn't relevant and I mis read it. Sorry about that, my error.

Sample rate is 44.1 or 48k in my book of digital so if you are clocking 44.056 or 47.97 that is to suit drop frame.

Well not sure what book that is but there are a ton of sample rates that are used in digital so... Since this forum has to do with video 44.1 really doesn't come into it so 48K and multiples are really all anybody reading this should be worried about.

You are off in your thinking about drop frame. Though for the UK maybe you are close enough.

DRop frame is TC the sample rates are, well, sample rates. Since audio is not recorded in frames sample rate and the frame rate of TC have no intrinsic connection. That and the fact that 29.97 and 29.97DF are EXACTLY the same speed, the ONLY difference being the counting sequence, means that have nothing in common.

For instance any broadcast show is going to be working in Drop Frame (in the US) because they need to know the "clock time" of the piece. IF they shot on video they would record 48K with DF TC. IF they shot on film and transferred to video then you would shoot at an over speed sample rate so when the tracks hit post and were played back at a standard rate they would be in sync with the footage that was now slowed down to "video speed", and use Drop Frame.

If you were shooting anything destined for film you don't use Drop Frame and except for the very rare case below would shoot at 48k

And I'm not sure where your SR come from they would only be used if you shot on video at a fractional frame rate and then transferred to film. Not unheard of but a very rare bird and in that case you wouldn't use Drop Frame anyplace in the flow. You also have a slowed down version of 44.1K and that would be even weirder. I don't know of anybody doing film work at 44.1. Some places in the 90's that did film, worked at 44.1 but not in decades. The reasoning with 44.1 in film was that Dolby Digital was 44.1 and CD sound libraries are 44.1 and SR conversion took forever so it was a time saver. Even then most shops worked at 48.

If it is irrelevant these days in the USA then why are we even discussing it.

Two reasons. First someone asked and second Drop Frame is not irrelevant, odd sample rates are largely irrelevant.

Surely HD in TV land is 1080i or 1080P depending if you want a video or film progressive look?

Those are screen resolutions not frame rates or TC rates.

Read Alan Roberts book as he is my reference point but sadly too many semi pro's are still doing stoopid things!

Probably not the right guy, the only Alan Roberts I found was the director/ writer of "The Happy Hooker Goes Hollywood".
Hey there are pro's doing stupid things also! ;~)

P.S I havent even mentioned field order and you should see some of the horror that comes thru the BBC these days with all sorts of muppets shooting with their DSLR's and go pros set at all sorts of frame rates and codecs.

Unfortunately you are not alone. I think low budget and broadcast are probably the hardest hit. Though we did get some weird stuff at times on big budget films, but usually things like unusable change notes or weird bits of sound a director or produced had on some wildly out dated media that the just absolutely HAD to use in the film.

 

[Noiz2]

As a bit of follow up on HD
Standard resolution/ frame rate options

[TABLE="class: wikitable, width: 822"]
[TR]
[TH="bgcolor: #F2F2F2, align: center"]Video mode[/TH]
[TH="bgcolor: #F2F2F2, align: center"]Frame size in pixels (W×H)[/TH]
[TH="bgcolor: #F2F2F2, align: center"]Pixels per image[SUP]1[/SUP][/TH]
[TH="bgcolor: #F2F2F2, align: center"]Scanning type[/TH]
[TH="bgcolor: #F2F2F2, align: center"]Frame rate (Hz)[/TH]
[/TR]
[TR]
[TD]720p[/TD]
[TD]1,280×720[/TD]
[TD]921,600[/TD]
[TD]Progressive[/TD]
[TD]23.976, 24, 25, 29.97, 30, 50, 59.94, 60, 72[/TD]
[/TR]
[TR]
[TD]1080i[/TD]
[TD]1,920×1,080[/TD]
[TD]2,073,600[/TD]
[TD]Interlaced[/TD]
[TD]25 (50 fields/s), 29.97 (59.94 fields/s), 30 (60 fields/s)[/TD]
[/TR]
[TR]
[TD]1080p[/TD]
[TD]1,920×1,080[/TD]
[TD]2,073,600[/TD]
[TD]Progressive[/TD]
[TD]24 (23.976), 25, 30 (29.97), 50, 60 (59.94)[/TD]
[/TR]
[TR]
[TD]1440p[/TD]
[TD]2,560×1,440[/TD]
[TD]3,686,400[/TD]
[TD]Progressive[/TD]
[TD]24 (23.976), 25, 30 (29.97), 50, 60 (59.94)

[/TD]
[/TR]
[/TABLE]

So even with HD the US is still dealing with fractional frame rates. Why, I can't imagine. Talk about pros making stupid mistakes ;~)

 

[GaryNattrass]

Ha ha I will agree with most of what you say scratch my head and thank god I live in PAL land 25p or 50i. :grin:

 

[ronin99]

It finally seems like I may get the mixed file from the "sound designer". 7 months for 5 minute film mix!!!

To try and get a definitive answer on this issue here's a little refresher of the situation.

The so called "sound designer", an amateur, can only work in 29.97 (yes, I know, hire someone else, a professional, I would if it was up to me and the person hired wasn't related to the boss), so is my workaround to take the mixed down WAV file i eventually will receive from this person into wave agent and make it 23.976? any other ideas on how to keep sync?

Film was edited in 23.976, a reference file of the film was exported at 23.976. The sound mixer then took the files into whatever program he used and can only work in 29.976 (so the film wasn't converted from 23.976, it was just placed in a 29.976 timeline).

He then did his mix, he then exports a .wav file from that timeline. That I will place back into my timeline that is 23.976.

Now if it all syncs up then great, but if it does not, is there any way through software to make the audio sync?

 

[Noiz2]

IF he played the video file at 29.97 then he was working to a picture that was speeded up by about 25%, so your option if that is the case is to slow down the WAV file. Yes it's possible but a 25% slow down is rather huge. I would check the file though since I suspect that his player was playing the video at or close to what it's metadata said it's frame rate was and he was just too inexperienced to know what was going on. His software may have not had a 23.976 option for his time line and he assumed the file was playing at that frame rate. As I'm sure it was mentioned earlier, sound has no frame rate. So the sound doesn't care what frame rate your video is at, but IF the video is played fast the editor would be placing sounds in the wrong place since the sound plays in "real time". But I suspect that the video was played back at it's own "speed" and your OK.

BTW it's 29.97 even, no third decimal place.