How to programatically test for audio sync

Question

I have a multimedia application that among other things converts video using FFMpeg. Video conversion being the pain that it is, I have in my test suits some tests that check our ability to convert various video formats, with emphasis on sample videos known not to work.

A common problem we've noticed from users is that some videos end up with their audio desynched after being processed, and I am looking for a way to check this in my tests.

Extracting the audio portion of the resulting videos is not a problem.

My best idea so far would be to check the offset of the first non-silence at both the beginning and end and compare each between the two videos, but I'm hoping someone smart has a better idea.

The application language/environment is Java, but since this is for testing, I'm free to use any toolset.

Thanks!

Answer 1

The basic problem is likely that the video and audio are different lengths. Extract the audio and test its length vs. the video length. If they are significantly different (more than maybe .05 sec, I'm not really sure what is detectable as "off"), then there's a problem.

To fix it, re-encode the audio to match the video length, and then put the audio and video back into a container format.