Displaying images with a high frame rate

Question

Hi,

here's the problem: I have a custom hardware device and I have to grab images from it in C#/WPF and display them in a window, all with 120+ FPS.

The problem is that there is no event to indicate the images are ready, but I have to constantly poll the device and check whether there are any new images and then download them.

There are apparently a handful of ways to do it, but I haven't been able to find the right one yet.

Here's what I tried:

• A simple timer (or DispatcherTimer) - works great for slower frame rates but I can't get it past let's say, 60 FPS.

• An single threaded infinite loop - quite fast but I have to put the DoEvents/it's WPF equivalent in the loop in order for window to be redrawn; this has some other unwanted (strange) consequences such as key press events from some controls not being fired etc..

• Doing polling/downloading in another thread and displaying in UI thread, something like this:

   new Thread(() =>
          {
              while (StillCapturing)
              {
                  if (Camera.CheckForAndDownloadImage(CameraInstance))
                  {
                      this.Dispatcher.Invoke((Action)this.DisplayImage);
                  }
              }
          }).Start();
  

  Well, this works relatively well, but puts quite a load on a CPU and of course completely kills the machine if it doesn't have more than one CPU/core, which is unacceptable. Also, I there is a large number of thread contentions this way.

The question is obvious - are there any better alternatives, or is one of these the way to go in this case?

Update:
I somehow forgot to mention that (well, forgot to think about it while writing this question), but of course I don't need all frames to be displayed, however I still need to capture all of them so they can be saved to a hard drive.

Update2: I found out that the DispatcherTimer method is slow not because it can't process everything fast enough, but because the DispatcherTimer waits for the next vertical sync before firing the tick event; which is actually good in my case, because in the tick event I can save all pending images to a memory buffer (used for saving images to disk) and display just the last one.

As for the old computers being completely "killed" by capturing, it appears that WPF falls back to software rendering which is very slow. There's probably nothing I can do about.

Thanks for all the answers.

Answer 1

I think you're trying for too simplistic of an approach. Here's what I would do.

a) put a Thread.Sleep(5) in your polling loop, that should allow you to get close to 120fps while still keeping CPU times low.

b) Only update the display with every 5th frame or so. That will cut down on the amount of processing as I'm not sure that WPF is made to handle much more than 60fps.

c) Use ThreadPool to spawn a subtask for each frame that will then go and save it to the disk (in a seperate file per frame), that way you won't be as limited by disk performance. Extra frames will just pile up in memory.

Personally I would implement them in that order. Chances are a or b will fix your problems.

Answer 2

You could do the following (all psuedocode): 1. Have a worker thread running dealing with the capture process:

List<Image> _captures = new List<Image>();
 new Thread(() =>
    {
        while (StillCapturing)
        {
            if (Camera.CheckForAndDownloadImage(CameraInstance))
            {
                lock(_locker){_captures.Add(DisplayImage);


            }
        }
    }).Start();

1. Have the dispatcher timer thread take latest captured image (obviously it will have missed some captures since last tick) and display. Therefore, UI thread is throttled and doing as little as possible, it isn't doing all the "capturing", this is done by worker threads. sorry I can't get this bit to format (but you get the idea):

   void OnTimerTick(can't remember params)
   

   { Image imageToDisplay; lock(_locker){imageToDisplay = _captures[k.Count - 1]; DisplayFunction(imageToDisplay); }

2. it might be that the list is a queue and another thread is used to bleed the queue and write to disk or whatever.