How can I optimize my screencasting utility?

Question

I'm developing a screencasting utility in C++.

It basically captures desktop frames and creates an AVI file. The algorithm is as follows:

1. Create a thread: this->m_hThread=CreateThread(NULL,0,thScreenCapture,this,0,NULL);

2. Capture desktop in thScreenCapture n times per second (like 5 fps).

   obj->Capture();

3. In Capture(), append the bitmap data to the avi file.

   this->appendBitmapToAvi(this->avifile, bmp);

This utility also records sound. So, in method thScreenCapture, sound data is also being appended to the avi file.

The problem is that a lag occurs between frames and sound when more than 6 frames (this can change depending on hardware configuration) are captured per second.

I'm seeking for a solution to optimize the algorithm. A solution may be buffering frames in memory, not appending all of them to the avi file on-the-fly. But this makes the code more complex because I have to deal with the sound data that's being captured in a different thread.

What do you suggest to increase the fps value that this utility supports without losing synchronization?

Answer 1

You could take a look at the source for other screencasting software, such as CamStudio, to see how they are doing it.

If your program is disk bound (and I suspect it is), then things might improve by using compression (this is how the big name programs, such as Camtasia Studio, operate)

Answer 2

What OS are you targeting? If tyou are working on Windows XP I would take a look at some of the DirectShow code at http://tmhare.mvps.org/downloads.htm, specifically Filter Graph Library.

Answer 3

Use a circular double or triple buffer to store the bitmap and sound each frame and use a separate thread to add the bitmap and sound to the avi. So data-collection is in one thread, data is in a circular (thread-safe) buffer and the data-storage is in another thread.

Answer 4

Are you writing the AVI file yourself? A noble effort, but there are APIs to help with this task.

If you're working on the windows platform, I'd suggest considering using the DirectShow or Media Foundation APIs to mux the audio and video to disk. DirectShow is the API for A/V capture, streaming and muxing on the windows platform.

This article on CodeProject talks about audio & video sync issues and the mechanism that DirectShow uses to overcome this difficulty.

Essentially, a reference clock is employed and the frames are timestamped.

There's a very active DirectShow community that is an extremely useful resource for new folks. TMH's website is well worth checking out - he's an MS MVP and is an active member of the community.

I hope this helps!