How to quickly acquire and process real time screen output

Question

I am trying to write a program to play a full screen PC game for fun (as an experiment in Computer Vision and Artificial Intelligence).

For this experiment I am assuming the game has no underlying API for AI players (nor is the source available) so I intend to process the visual information rendered by the game on the screen.

The game runs in full screen mode on a win32 system (direct-X I assume).

Currently I am using the win32 functions

#include <windows.h>
#include <cvaux.h>
class Screen {
    public:
    HWND    windowHandle;
    HDC     windowContext;
    HBITMAP buffer;
    HDC     bufferContext;
    CvSize  size;
    uchar*  bytes;
    int channels;
Screen () {
    windowHandle = GetDesktopWindow();
    windowContext   = GetWindowDC (windowHandle);
    size = cvSize (GetDeviceCaps (windowContext, HORZRES), GetDeviceCaps (windowContext, VERTRES));
    buffer = CreateCompatibleBitmap (windowContext, size.width, size.height);
    bufferContext = CreateCompatibleDC (windowContext);
    SelectObject (bufferContext, buffer);
    channels = 4;
    bytes = new uchar[size.width * size.height * channels];
}

~Screen () {
    ReleaseDC(windowHandle, windowContext);
    DeleteDC(bufferContext);
    DeleteObject(buffer);
    delete[] bytes;
}

void CaptureScreen (IplImage* img) {
    BitBlt(bufferContext, 0, 0, size.width, size.height, windowContext, 0, 0, SRCCOPY);
    int n = size.width * size.height;
    int imgChannels = img->nChannels;

    GetBitmapBits (buffer, n * channels, bytes); 

    uchar* src = bytes;
    uchar* dest = (uchar*) img->imageData;
    uchar* end  = dest + n * imgChannels;

    while (dest < end) {
        dest[0] = src[0];
        dest[1] = src[1];
        dest[2] = src[2];

        dest  += imgChannels;
        src += channels;
    }
}

The rate at which I can process frames using this approach is much too slow. Is there a better way to acquire screen frames?

Answer 1

As a general method, I would hook the buffer-flipping function calls so I can capture the framebuffer on every frame.

http://easyhook.codeplex.com/