Time-passage callback when rendering recorded and live/streaming audio/video.

Question

Guys...! I'm a complete newbie to this area of audio/video formats/codecs, their players, their exposed APIs, etc. Would greatly appreciate your patience as well as advice on this question of mine.

Question: Almost all audio/video players show time-passage information in hh:mm:ss format. I want to be able to register a callback of mine -- which could either be a Java/applet callback or a Javascript callback -- which gets called at every 'time tick' and runs my custom code. Perhaps it's not a time tick but a 'frame-passage' event in the lingo of codecs. Perhaps, these codecs/players allow you to even specify the frequency of callback (every so many frames or seconds)...

Secondly, I wish to be able to do this for both recorded content as well as live/streaming content.

What opensource software, tools, APIs, 'concepts', etc I should be looking at? I'd like to be able to handle the most popular formats such as flv, avi, mp3, mpeg. Ideally, I would prefer to deal with something in Java / Javascript.

Not sure where to start. A sample, a code snippet, or even just the process/steps I need to follow would be a great help. Thanks again.

Answer 1

This depends on what kinds of environment you are running in and the kinds of things you want to do in your callback. Some multimedia frameworks have a way to add video, subtitle, or audio filters, which have access to the various data from the stream. One such open source framework is GStreamer. Another is the Java-based Processing language, although it cannot directly decode popular video formats on its own. I think that VLC, which of course is the open source king of video formats, also has an API that could potentially meet your needs.

The new HTML5 video element defines a timeupdate event, which will fire every 15 to 250ms while the video is playing. The currentTime attribute can be accessed to get the current playback position. It is also possible to manipulate the HTML5 video directly in JavaScript code.

If you just want a time callback while the video is playing and do not need to filter the data then you could just use the standard timer facilities of your language, e.g. setTimeout() in JavaScript, to request a timer callback at the frame rate of the video.