Display image in second thread, OpenCV?

Question

I have a loop to take in images from a high speed framegrabbger at 250fps.

/** Loop processes 250 video frames per second **/
while(1){
  AcquireFrame();
  DoProcessing();
  TakeAction();
}

At the same time, I would like the user to be able to monitor what is going on. The user only needs to see images at around 30 fps (or less). How do I set up a second thread that displays the current frame every so often?

Thread(){
  cvShowImage();
  Wait(30); /** Wait for 30 ms **/
}

I am on Windows on a quad core Intel machine using MinGW, gcc and OpenCV 1.1. The main criteria is that the display thread must take as little time away from my main processing loop as possible. Every millisecond counts.

I have tried using CreateThread() to create a new thread with cvShowImage() and cvWaitKey() but apparently those functions are not threadsafe.

I am considering using OpenMP, but some people report problems with OpenMP and OpenCV. I also am considering trying to use DirectX directDraw because apparently it is very fast. but it looks complicated and evidentally there are problems using Windows DLL's with MinGw.

Which of these avenues would be the best place to start?

Answer 1

Since the frame grabbing doesn't need to use the UI, I'd set up a secondary thread to handle the frame grabbing, and have the original thread that handles the UI display the sample frames. If you tried to display the frame currently be grabbed, you'd have to lock the data (which is generally fairly slow). To avoid that, I'd display a frame one (or possibly two) "behind" the one currently being grabbed, so there's no contention between grabbing and displaying the data. You'll still have to ensure that incrementing the current frame number is thread-safe, but that's pretty simple -- use InterlockedIncrement in the capture thread.

Answer 2

Isn't it just enough to just use an if condition and a "count" variable upto 250 to display every 8th capture (250/30)?

Answer 3

I'm sorry I can't give you a better answer right now, but it seems that your question is not about the structure of your program but rather about the tool you should use to implement multithreading. For this I would recommend Qt. I have been using Qt for a while but I'm just now getting into multithreading.

It seems to me that your best bet might be a QReadWriteLock. This allows you to read from an image but the reader thread will give up its lock when the writer thread comes along. In this case you could keep a copy of the image you last displayed and display it if the image is locked for writing.

Sorry again that I can't be more detailed but, like I said, I'm just getting into this as well. I'm basically trying to do the same thing that you are, but not nearly as fast :). Good luck!

Answer 4

Ok. So embarrassingly my question is also its own answer.

Using CreateThread(), CvShowImage() and CvWaitKey() as described in my question actually works-- contrary to some postings on the web which suggest otherwise.

In any event, I implemented something like this:

/** Global Variables **/
bool DispThreadHasFinished;
bool MainThreadHasFinished;
iplImage* myImg;

/** Main Loop that loops at >100fps **/
main() {
  DispThreadHasFinished=FALSE;
  MainThreadHasFinished=FALSE;
  CreateThread(..,..,Thread,..);

  while(IsTheUserDone()){
    myImg=AcquireFrame();
    DoProcessing();
    TakeAction();
  }
  MainThreadHasFinished=TRUE;


  while (!DisplayThreadHasFinished){
     CvWaitKey(100);
  }

  return;
}

/** Thread that displays image at ~30fps **/
Thread(){
  while (!MainThreadHasFinished) {
    cvShowImage(myImage);
    cvWaitKey(30);
  }
DispThreadHasFinished=TRUE;
return;
}

When I originally posted this question, my code was failing for unrelated reasons. I hope this helps!