Detecting video playing in browser from a screenshot -- OpenCV

Question

I would like to draw a rectangle around a video playing on my screen. For example, I am watching a YouTube video in my browser. I would like to be able to take a screenshot, analyze that screenshot, and then draw a rectangle around where the YouTube video is playing. I have just started looking into how I might be able to to this. I came across OpenCV. I understand that OpenCV covers many computer vision techniques. Would any of them be particularly well suited for this task? Also, is this something that can be done in real time? Finally, is there a technique that would work for both in browser and full screen?

Thanks!

Edit:

Marc makes a good point. Ideally I would like to be able to identify the video using just one screenshot. I am not sure if that if possible though. How about comparing screenshots a taken a second or two apart?

Answer 1

If it is just a Youtube video you could use an edge detection algorithm (such as Canny or Sobel) to find the boundary of the video player, and look for the Youtube logo using OpenCV MatchTemplate method. A combination of those 2 should give you pretty good accuracy.

You can also check for the boundary rectangle size, since Youtube plays at certain height x width windows.

Answer 2

If the video is not playing in full-screen, you can:

1. Detect moving parts (simple optical flows
2. Perform template matching for the video controls :D (much simpler than YouTube logos, ...)
3. Morphological operations on the flow fields.

If the video is playing in full-screen, an SVM trained on the histogram of optical flow should tell you that it is full-screen.

Answer 3

You could also look at the amount of texture detail, particularly in each color.

Computer screens are generally full of blank areas. Where there is texture (ie edges) then it's generally a single color (ie text) a video or any photo image is generally more complex and consists of many more colors.