DirectShow analog for Symbian OS?

Question

So Windows Mobile has its DirectShow for media editing capturing and so what is its analog for Symbian OS (s60 v5)?

Answer 1

There are a number of APIs on Symbian which provide audio & video functionality; note, I'm not familiar with DirectShow but based on a brief look at Wikipedia it looks like the DirectShow APIs are more powerful that what Symbian (currently) offers.

Here's a brief overview of the APIs available of Symbian and what you can do with them.

DevSound

The DevSound API, CMMFDevSound, is the lowest level audio API available. It provides audio input and output streaming in a variety of formats. The exact formats supported will depend on the specific hardware you're using, but it always supports 16 bit PCM and usually supports AMR, AAC, MP3 too. The API is not the easiest to use but probably worth using if you want to perform 'real-time' audio streaming.

The classes CMMFAudioInputStream and CMMFAudioOutputStream are essentially wrappers around DevSound; they are easier to use but not quite as powerful.

Audio Clip utilities

For basic file (clip) playback/recording, you can use CMdaAudioPlayerUtility and CMdaAudioRecorderUtility. These provide functionality at the level of 'open file', 'play', 'seek', etc. They are the easiest option if you just want to play/record audio files. They are no good if you want to be able to generate audio as it plays, or process it as it is recording.

Video Clip Utilities

These are the analog of the audio clip utilities, but used for playing/recording video files. The classes to use are CVideoPlayerUtility and CVideoRecorderUtility. They involve using Direct Screen Access and/or ECAM to render/record video (see below).

Direct Screen Access

Direst Screen Access (DSA) provides fast access to the video hardware, and is used for rendering video, camera view finders, or anything else that requires high speed graphics. The class to start with is CDirectScreenAccess.

The exact behaviour of DSA can very depending on your specific hardware: for example, some implementations will 'abort' your DSA if a menu or another window is drawn on top of the DSA area; others may allow the DSA to continue in the background, performing clipping og your DSA region in hardware.

Note, if you are using DSA directly to render video, you will need to arrange for the video stream to be decoded yourself; DevVideo (below) may help here. Only uncompressed bitmaps can be rendered using DSA.

ECAM

The ECAM API, CCamera, provides access to any camera hardware present. It can provide live uncompressed video frames from the camera (for encoding or to render a viewfinder, usually using DSA), or it can take snapshots.

DevVideo

The DevVideo API provides low level access to video encoding and decoding hardware. Depending on the specific hardware you're using, it may also support rendering of an encoded video stream to a DSA region. The API is difficult to use and you are likely to encounter irregularities between different phone models. I would only recommend using it if you really need to perform hardware accelerated streamed video encoding/decoding. Note, the video clip utilities mentioned above use DevVideo under the hood, so you will benefit from any hardware acceleration present if you use those.

The classes to look at are CMMFDevVideoPlay and CMMFDevVideoRecord. Some caveats:

• Nokia have excluded the .lib files from some SDKs meaning you can't use these APIs with the standard SDK; I'm not sure what the current situation is here.
• Realistically, to get these classes working, you will probably need professional help from Nokia, i.e. you will need to pay for it. I would glad to be proved wrong here :-)

Which APIs to use?

Good question. It depends what you want to do. If all you want to do is basic playback/recording of audio/video, use the clip utilities. They are much easier to use that the others, if they are powerful enough for your needs.

If, however, you need to perform streaming of audio or video, you'll need to use DevSound and/or DSA. If you want to do something like:

• render audio and video being streamed over a network
• generate audio in real-time, for example game sound effects
• process recorded audio/video in real time, for example streaming from the camera over a network

Then you'll need to use the low level APIs, and you probably have a lot to learn!

Some links

• some example source code on the Symbian books page. The code from Symbian OS C++ for Mobile Phones v3 contains examples for all the higher level multimedia APIs (i.e. not DevSound or DevVideo). Unfortunately, the book itself is not available online.
• Forum Nokia has some example code which may be worth a look.