How to render video from raw frames in WPF?

Question

I have a special video camera (using GigEVision protocol) that I control using a provided library. I can subscribe to a frame received event and then access the frame data via a IntPtr.

In my old WinForms app I could render the frame by either creating a Bitmap object from the data and setting it to a PictureBox Image, or by passing the PictureBox handle to a function in the provided library which would draw directly on the area.

What is the best and fastest way to do a similar thing in WPF? The video camera runs anywhere from 30 to 100 fps.

edit(1):

Since the frame received event is not on the UI thread it has to work across threads.

edit(2):

I found a solution using WriteableBitmap:

void camera_FrameReceived(IntPtr info, IntPtr frame) 
{
    if (VideoImageControlToUpdate == null)
    {
        throw new NullReferenceException("VideoImageControlToUpdate must be set before frames can be processed");
    }

    int width, height, size;
    unsafe
    {
        BITMAPINFOHEADER* b = (BITMAPINFOHEADER*)info;

        width = b->biWidth;
        height = b->biHeight;
        size = (int)b->biSizeImage;
    }
    if (height < 0) height = -height;

        //Warp space-time
        VideoImageControlToUpdate.Dispatcher.Invoke((Action)delegate {
        try
        {
            if (VideoImageControlToUpdateSource == null)
            {
                VideoImageControlToUpdateSource =
                    new WriteableBitmap(width, height, 96, 96, PixelFormats.Gray8, BitmapPalettes.Gray256);
            }
            else if (VideoImageControlToUpdateSource.PixelHeight != height ||
                     VideoImageControlToUpdateSource.PixelWidth != width)
            {
                VideoImageControlToUpdateSource =
                    new WriteableBitmap(width, height, 96, 96, PixelFormats.Gray8, BitmapPalettes.Gray256);
            }

            VideoImageControlToUpdateSource.Lock();

            VideoImageControlToUpdateSource.WritePixels(
                new Int32Rect(0, 0, width, height),
                frame,
                size,
                width);

            VideoImageControlToUpdateSource.AddDirtyRect(new System.Windows.Int32Rect(0, 0, width, height));
            VideoImageControlToUpdateSource.Unlock();
        }
        catch (Exception ex)
        {
            Console.WriteLine(ex.ToString());
        }
    });
}

In the above, VideoImageControlToUpdate is a WPF Image control.

For more speed I believe the VideoRendererElement found on codeplex is faster.

Answer 1

I hope that this blog post provides the solution you need.

Answer 2

Best way: WriteableBitmap.WritePixels(..., IntPtr source, ...)

Fastest way: Use WIC and all operations in IntPtr unmanaged memory. But why use WPF at all in this case? Consider using DirectX overlay if that kind of performance is required.