Fast screen capture and video recording

Question

Has anyone worked on capturing screen to video stream (to store in local file or send to network)?

I understand how it can be done, and have several test solutions working - but we have trouble achieving decent performance. We need to capture about 4 megapixels screen space of changing text and vector graphics, on a computer where CPU is already heavily utilized.

Acceptable (though far from desired) performance is achieved by sending uncompressed BMP frames to network, but for many reasons at least some compression on-site is important.

Any suggestions on how to encode using as little processing power as possible: May be a very fast codec? Or some tricks to avoid copying images in memory? Is capturing screen with DirectX (most of screen is in WPF) worth doing?

Answer 1

many softwares does that with a graphical interface like xvidcap or camstudio, but ffmpeg may be have a good solution for you ...

Answer 2

Ok... this is a wild guess because I've never tried it... but it seems plausible. I think you should use Nvidia CUDA. For example:

I was thinking you can create textures from the image (in memory) and compress it afterward. In the CUDA SDK there is a sample for DirectX Texture Compressor (DXTC):

High Quality DXT Compression using CUDA. This example shows how to implement an existing computationally-intensive CPU compression algorithm in parallel on the GPU, and obtain an order of magnitude performance improvement.

You can store a number a textures in memory (depending of the amount of video memory) and write them to disk/socket on another thread.

This is only a suggestion... I think the best way is to implement an encoding algorithm (see TMPGEnc) using CUDA to move the load from the CPU to GPU, but this is tricky and requires a lot of work.

Answer 3

I just ran into this while searching for CUDA and screen capture and thought I should add my experience. I've created a solution in the past using VNC and FFMPEG. If you have a look at the VNC protocols, you'll see that it does its transmissions based off of delta windows with a new image. Basically, previous screen + changes = new screen. The only thing that needs to be transmitted are the changes. You'll find lots of tricks to minimize transmission cost and lots of different payload extensions to transfer the data and it's a great resource even if you decide to roll your own with the knowledge gained. Once we were using VNC to move the pixel data, we found that raw pixel data was more expensive to our cpu than the jpeg data because the buffer copies were more expensive than the compression.