dual sound out of sync changes in time

[den59]

Dear All,

It is a first time i try to edit dual sound interview, one was recorded on camera (via on-camera shotgun in GH3) and the second - external cheap Olympus recorder (in WMA).
I'm on mac, so I have to convert WMA to something else, i used All2MP3 to make an MP3 out of it.
When I look into those tracks (on-camera + mp3) in Premiere CS6 and try to sync them using Plural Eyes, I have an annoying problem:
Once I sync the beginning of the interview, at some point later it become out of sync.
I wonder what is the reason for this? Could WMA->MP3 introduce that, what do you guys use to import WMA to CS6 on Mac?
Or would you say it's because of a cheap audio recorder which "distorts" the time while recording?
It could also be an issue with the camera, but I never saw anyone complaining about such problems for DSLRs.

p/s/ next time i will replace my recorder with something better, but for now i just have to make things work.

EDIT1: How to manually correct audio drift in Premiere Pro CS6

I found the following procedure for eliminating audio drift in premiere cs6 to work best for me:
1. Using waveform monitor put markers in both separate audio track and video track at two equivalent points between which the audio is to be synchronized (that is 2 markers for each audio source).
2. Put both tracks at the same sequence. Trim both according to the markers. If there is a drift the total duration will be different.
3. Select both, right click ->synchronize->marker -> (choose the first one).
4. Use rate stretch tool to make the total duration of the separate audio track equal to the video track.

Regards,
Denis

 

[Steve House]

Sync "drift" is basically caused by the sample clocks in the camera and the recorder running at slightly different rates. One of the reasons pro recorders cost what they do is the engineering and components that go into making accurate and stable clocks costs $$ so one source of your problem is trying to use a consumer audio note-taker instead of a more capable recorder. Then you're compounding the problem by recording in WMA, a highly compressed lossy format that can further introduce timing issues of its own. THEN you're converting THAT to MP3, yet another lossy compressed format that can introduce its own issues. WMA and MP3 are distribution formats, not mastering formats, and should never be used for original recordings. Audio for video should be recorded in an uncompressed format such as WAV or a lossless compressed format such as FLAC, 48kHz sample rate, 16 or 24 bit depth. Yes, compression can save you storage by giving you longer recording times but that's its only advantage and memory is cheaper than dirt these days. Trying to squeeze those extra minutes onto a memory card is being penny wise but pound foolish.

 

[TheDingo]

As Steve stated, sync "drift" is very common in lower cost audio gear, so you need to learn how to re-sync your audio in post.

I often shoot very long university/corporate events where I may have to sync 4 hours of audio with footage, so I've learned a couple of different ways to do this very quickly.

Method 1

Using the time-stretch/compress feature built-in to the Sony Vegas Pro video editor, I sync up the beginning of the long audio/video clips and trim the two tracks so they start at the same place. Then I do the same thing to the end of the two clips, and finally use the time-stretch/compress feature in Vegas ( CTRL Mouse-drag ) to adjust the length of the audio track to match the length of the video clip. This can be done in less than a minute for hours of footage.

Method 2

Do the same thing as above, but use the Time-Warp function built-in to the $19 GoldWave sound editor that allows you to adjust the length of your audio track down to 0.0001 seconds, save the new adjusted audio file and bring it back into my video editor. This takes a little longer than the Vegas method ( 2-3 minutes instead of less than 1 minute ), but the results are just as good.

To make life even easier you can front and tail slate your footage, so you will have easy to find and match sync points at the start and end of your recording.

It's really quite easy to do once you get the hang of it.

 

[Noiz2]

I will second (third) all the above. Your recorder is not likely to have a stable clock, nor one that is particularly accurate. Compressed formats are not very timing stable. Conversion always introduces the possibility of a timing shift. And if everything in the audio chain was perfect your camera is also not fantastically stable timing wise either.

All that said if you have a consistent drift and it's around three frames a minute you may have a conversion error. where at some point one of the conversions did a "frame rate conversion" that it didn't need to do.

What you should expect from a non $$$ setup is some small drift. Usually it's not enough to worry about unless your takes are longer that three minutes or so. If your results are worse than that either something is wrong in the work flow or some piece of equipment is not up to working in sync.

 

[TheDingo]

For the record, the sync drift I experience with low cost gear is usually in the range of 6-12 frames per hour of recording, and I always try to set the audio recorder to use the same sampling rate as the audio in the video camera, which is usually 48 kHz at 16 or 24 bits.

 

[puredrifting]

Tascam DR40 and Canon 5D MKII. I can go about 5-6 minutes before getting one frame of drift. Luckily, in this case, the 5D MKII only does 12 minute takes so I never have to slip the Tascam audio the one fram more than once per video clip. If you are getting a lot of sync drift, chances are you have have a sample rate mismatch.

 

[den59]

Thank you all for all the answers. It really helps.
Although, I am not quite sure why would sampling rate mismatch cause additional problems?
Should not editing software handle different sampling of audio with no problems (assuming that 1 second in each sampling is indeed 1 second and not shorter/longer)?
If analogy to resolution holds, one does not care about two photos having different resolution as long as both represent the same "dimension/size/...". One can always stretch them to fit each other perfectly.
And audio sampling rate is nothing else but the way to go from continuous to discrete signal, right?

@Steve,
i did record to WMA cos that el cheapo recorder (damn it!) did not have any other options, i guess it would be better even with my iPhone and Rode app recording a signal from lav (idea they sell as Rode SmartLav). Not a pro solution, of course. But I wonder how better would some not-so-expensive recorders like Zoom H4N be, would I gain much compared to iPhone in terms of a drift? It will sure be better in terms of noise, no questions about it. I think I will try to do some tests to see what is the drift of GH3 + Rode app on iPhone...

@TheDingo
thanks for explaining your method to deal with this problem. I will have a look if something similar is available in CS6.
6-12 frames/hour is definitely better than what I have. I have a drift of about 24 frames in some 10 minutes.

@Noiz2
I understand that conversion of lossy WMA to MP3 could add to the problem, but I don't know any other way to use WMA on Mac in CS6. I also have a Windows system, maybe I should look if I can convert WMA to something that CS6 on Mac could use...

@puredrifting
how much does your setup drift, is it about the same 5-6 frames/hour TheDingo mentioned?


Maybe to continue on that question, what would you guys recommend to avoid this problem in future?

I'm a one-man-band with a camera+shotgun and a lavaliere mic.
Do you think it is better to by a mixer/pre-amp/recorder (i quite like Tascam DR-60D apart from it producing an audible clicks when running levels) + Sennheiser G3 wireless + shotgun and record both at the camera and recorder? Hopefully the result won't have a big drift in some 15-20 minutes interviews.
Problem with this solution is that it costs about 1000 Euro here in Europe.

Since I do it for hobby, I would prefer some cheaper solutions which give an O.K. quality. I am already happy if it does not screw up the whole thing (as what happened with this el cheapo Olympus recorder). Something like Tascam DR40 is a nice thing to have, but I'm not sure if it is convenient to use it with lavs? It is quite big, it seems.

 

[TheDingo]

I am not quite sure why would sampling rate mismatch cause additional problems?

It's not the sampling frequency, but rather it's because the timing clocks ( crystal oscillator ) of each device runs at a different frequency so they will drift apart over time. The expensive pro gear enables you to sync the clocks so everything runs at the same timing frequency.

I have a drift of about 24 frames in some 10 minutes.

This shouldn't be a problem to fix as long as the drift amount is consistent. ( linear drift can easily be fixed )

What would you guys recommend to avoid this problem in future?

Learn to correct drift ( it's very easy ), and use in camera audio when it's applicable. ( the JuicedLink Riggy units are a good fit with the GH3 camera, along with Sennheiser G3 UHF Lav systems )

 

[Noiz2]

Although, I am not quite sure why would sampling rate mismatch cause additional problems?
Should not editing software handle different sampling of audio with no problems (assuming that 1 second in each sampling is indeed 1 second and not shorter/longer)?
If analogy to resolution holds, one does not care about two photos having different resolution as long as both represent the same "dimension/size/...". One can always stretch them to fit each other perfectly.
And audio sampling rate is nothing else but the way to go from continuous to discrete signal, right?

Sample rate shouldn't matter but conversions are not all equal. So basically even though it shouldn't be a problem anything that avoids a conversion is a good choice.

@Steve,
i did record to WMA cos that el cheapo recorder (damn it!) did not have any other options, i guess it would be better even with my iPhone and Rode app recording a signal from lav (idea they sell as Rode SmartLav). Not a pro solution, of course. But I wonder how better would some not-so-expensive recorders like Zoom H4N be, would I gain much compared to iPhone in terms of a drift? It will sure be better in terms of noise, no questions about it. I think I will try to do some tests to see what is the drift of GH3 + Rode app on iPhone...

I would say probably the zoom would beat the iPhone. You want something that was designed to record accurately. The iPhone is, well, a phone not a recorder. You can ride a bicycle on a freeway, but it's not a good idea.

@TheDingo
thanks for explaining your method to deal with this problem. I will have a look if something similar is available in CS6.
6-12 frames/hour is definitely better than what I have. I have a drift of about 24 frames in some 10 minutes.

That is a huge drift. Your problem with correcting it by time stretching is that you will probably hear the pitch change. There are stretchers that will pitch correct but some don't do a very smooth job of it so??? I would do some tests and see what sounds best.

@Noiz2
I understand that conversion of lossy WMA to MP3 could add to the problem, but I don't know any other way to use WMA on Mac in CS6. I also have a Windows system, maybe I should look if I can convert WMA to something that CS6 on Mac could use...

I understand you had to record in a compressed format but you should have converted to WAV or AIFF for editing. MP3 is an awful format for editing. Like h.264 editing requires that the file be uncompressed edited and recompressed. It may all happen under the hood so you don't realize it but you end up recompressing your sound over and over again when you edit and it gets a little worse each time

Maybe to continue on that question, what would you guys recommend to avoid this problem in future?

Get a better recorder and don't use compressed formats. May not solve the problem but should get you a lot closer.

Since I do it for hobby, I would prefer some cheaper solutions which give an O.K. quality. I am already happy if it does not screw up the whole thing (as what happened with this el cheapo Olympus recorder). Something like Tascam DR40 is a nice thing to have, but I'm not sure if it is convenient to use it with lavs? It is quite big, it seems.

"Voice recorders" have very poor stability because they don't need it for what they are used for. The DR60 is nice the DR40 is smaller than the DR60 (a touch longer and slightly narrower but less than half as thick. Go look at a Nagra V if you want to see big.

The 60 has more inputs but the 40 has built in mics so you can use it on it's own for some things and it's smaller and lighter. Though if your hanging off a DSLR then the 60 will be more convenient. Not sure why you think it would have problems with lavs?

 

[den59]

Thanks for your replies, Noiz2 and TheDingo.

@Noiz2:
with regards to lavs i mean wired lavs, for this i suppose a recorder needs to be small and thin to fit into a pocket or something.


Anyway, here is a nice tutorial i found on how to fix the drift in Premiere. (should work for any other editing software as well).
In a nutshell, one finds two eqvivalent points in both audio track from video and a separate audio track. Next, one calculates exactly the difference between those two points in frames.
From this mismatch a separate audio track is made to play longer/sorter so that the difference between the two points is equal for both audio tracks.

I still have problems with drift myself, could be because the drift is not constant. But in theory the tutorial should work and should be robust.

 

[Noiz2]

Anything you record un monitored is a total crap shoot and you should assume it won't be usable, so I would not be going to a recorder in someones pocket except under duress.

All picture editing programs work in FRAMES so you are never going to reliable get better than 1/2 a frame in sync. Ironically actual film has four sprocket holes per frame (35mm) so syncing in film is actually better than in our new NLE world.

The two issues with what they are doing in Premiere are that as I said they are working in frames and 1/4 frame is noticeable. They are assuming a constant speed error and a lot of times with cheap recorders that is not the case. If the offset is not really small you are going to get a noticeable pitch shift by stretching or squishing the audio.

It's not always possible or convenient but you should always try to work on things with the tools that were designed to work with that media. NLE's are picture editors with some sound capabilities. DAW's are sound editors with some video capabilities. They will each do a better job working on what they were designed to work with.

 

[den59]

@Noiz2,

I don't quite get your point "Anything you record un monitored is a total crap". That a bit contradictory of many advices here to use wired lavs when there is no money for wireless. It does not mean audio levels won't be tested before hitting record button, but for obvious reason one can't change levels while recording wired lavs. Do you mean to say that wired lavs should never be used?

 

[Noiz2]

@Noiz2,

I don't quite get your point "Anything you record un monitored is a total crap". That a bit contradictory of many advices here to use wired lavs when there is no money for wireless. It does not mean audio levels won't be tested before hitting record button, but for obvious reason one can't change levels while recording wired lavs. Do you mean to say that wired lavs should never be used?

?Why on earth can you not change levels on wired lavs? Testing the levels before recording only gives you a ballpark. Actors don't "perform" while your checking, no mater what they say the levels will be different 99% of the time. You also won't know if the mic is in a bad location and getting rubbed by something etc.

And that was a total "crap shoot" ie gamble, no idea if what you get is going to work or not.

 

[Steve House]

@Noiz2,

I don't quite get your point "Anything you record un monitored is a total crap". That a bit contradictory of many advices here to use wired lavs when there is no money for wireless. It does not mean audio levels won't be tested before hitting record button, but for obvious reason one can't change levels while recording wired lavs. Do you mean to say that wired lavs should never be used?

There's absolutely NO 'obvious reason you can't change levels while recording wired lavs' because that's not at all true. You put your lavs on the talent and run them into a mixer. The mixer output goes to your camera's or your recorder's line input. The camera's monitor output goes back to the mixer's monitor input. The sound recordist rides gain using the mixer's controls and monitors the resulting recording. A well-designed mixer allows the operator to selectively listen to either the mix leaving the mixer bound for the camera or the return back from the camera monitor output as required. The reason for the frequent advice to use wired lavs is it takes hundreds, if not thousands, of dollars investment for wireless to get the same audio quality and reliability you can get with $20 worth of cable. But the technique for adjusting levels while recording is the same for both.

Your post also seems to suggest that monitoring isn't required with wireless lavs. Again, not true. Continuous monitoring of the signal is required for ALL audio: wireless lavs, wired lavs, boom mics, plant mics, you name it. If you're recording you should be monitoring.

 

[den59]

?Why on earth can you not change levels on wired lavs? Testing the levels before recording only gives you a ballpark. Actors don't "perform" while your checking, no mater what they say the levels will be different 99% of the time. You also won't know if the mic is in a bad location and getting rubbed by something etc.

And that was a total "crap shoot" ie gamble, no idea if what you get is going to work or not.

There's absolutely NO 'obvious reason you can't change levels while recording wired lavs' because that's not at all true. You put your lavs on the talent and run them into a mixer. The mixer output goes to your camera's or your recorder's line input. The camera's monitor output goes back to the mixer's monitor input. The sound recordist rides gain using the mixer's controls and monitors the resulting recording. A well-designed mixer allows the operator to selectively listen to either the mix leaving the mixer bound for the camera or the return back from the camera monitor output as required. The reason for the frequent advice to use wired lavs is it takes hundreds, if not thousands, of dollars investment for wireless to get the same audio quality and reliability you can get with $20 worth of cable. But the technique for adjusting levels while recording is the same for both.

Your post also seems to suggest that monitoring isn't required with wireless lavs. Again, not true. Continuous monitoring of the signal is required for ALL audio: wireless lavs, wired lavs, boom mics, plant mics, you name it. If you're recording you should be monitoring.

I fully agree and stand corrected.

I was just thinking about a one-man band (me) doing camera and monitoring shotgun simultaneously. I did not consider recording wired lav along with an on-camera shotgun.
So, in case of sit-down interviews, I might have something like "wired lav"->long cable->juiced link (mixer/amp)->camera + shotgun->juiced link->camera.
I suppose a long cable (lav->juicde link) will degrade quality, but it seems there is no other wired solution for my case. Do you think this set up will work, at least to some degree?

Obvious alternative is to go wireless with Sennheiser G3.

 

[mhood]

I suppose a long cable (lav->juicde link) will degrade quality, but it seems there is no other wired solution for my case.

I was taught you could run balanced XLR cable to the moon and back with no apparent loss. I don't think anything under 100' will be a problem for you.

 

[Steve House]

...
I suppose a long cable (lav->juicde link) will degrade quality, but it seems there is no other wired solution for my case. Do you think this set up will work, at least to some degree?...

A long BALANCED (usually XLR) cable is not going to present a signal degradation problem. If you were running a consumer mic with an unbalanced cable it could be an issue but balanced cable runs of several hundred feet or even more are not at all uncommon.

Ig you're puttinmg lavs on your subject and recording it in-camera why would you also record an on-camera shotgun at all?

 

[den59]

I was taught you could run balanced XLR cable to the moon and back with no apparent loss. I don't think anything under 100' will be a problem for you.

great, thanks. will be looking at this solution.

A long BALANCED (usually XLR) cable is not going to present a signal degradation problem. If you were running a consumer mic with an unbalanced cable it could be an issue but balanced cable runs of several hundred feet or even more are not at all uncommon.

Ig you're puttinmg lavs on your subject and recording it in-camera why would you also record an on-camera shotgun at all?

Because I put it on interviewee and there is an interviewer who does not have a lav.

 

[Richard Crowley]

It is amusing that a new generation of video producers just assume that random pieces of equipment (of whatever quality level) should ipso facto always be in perfect sync. Of course in the Real World that NEVER happens. If you think about it, you have probably NEVER seen a wristwatch, clock, computer, car, appliance, etc. clock that kept "perfect time". They ALL require re-syncing to some standard reference at whatever interval is required for the desired accuracy. Certainly there are "atomic clocks" which automatically reset themselves, sometimes several times per day, by using a time/frequency standard broadcast signal: WWV in the USA, CHU in Canada, DCF77 in Germany (serving most of Europe), etc. And in modern times, GPS is used by many as a reliable source of time reference and available anywhere on the planet.

People have forgotten that they(we) used to do extraordinary things to keep the camera and audio recorder in sync. In early days a studio would have their own 3-phase generator that all the cameras and sound recorders would use for powering the motors. That is the origin of the response "Speed!" when the audio dubber was locked into the master phase reference. Then we had various schemes where the camera would generate a sync signal of some kind which was recorded on a separate track on the sound recorder. And then with the advent of the computer era, the famous "SMPTE TimeCode" and "Genlock".

As we have seen from this discussion, even modern pro-sumer gear is using mass-produced crystal oscillator clocks with increasingly good accuracy. But it would be silly to expect that ANY kind of gear could keep perfect sync for more than a few minutes at a time. In the past, I have found that "diddling" the VIDEO track was preferable to shrinking/stretching the AUDIO track. Perhaps newer versions of video NLE apps have improved their handling of making small changes to audio tracks, but I have heard terrible results from making very small changes. IMHO, it is much more acceptable to typical viewers to duplicate or drop a frame of video here or there vs. creating audible artifacts in the sound track.

Certainly we have seen occasional reports by people who happen to have a camera and an audio recorder that keep PERFECT sync for hours on end. That is not impossible, but it is very rare, at least IME. And it probably is not repeatable in unusually hot or cold situations, either.

 

[Steve House]

...Because I put it on interviewee and there is an interviewer who does not have a lav.

An on-camera shotgun is not going to be pointed at an off-camera interviewer, thus putting him off-mic for his voice. It would be better to use two lavs, one on each person. One lav goes to left channel, the other to right, and the stereo camera track is split into two mono tracks in post. Not that that has anything to do with sync drift issues.