Tracking Objects in Videos

Question

Are there any open Source libraries for tracking objects and over-laying graphics in videos ? Specifically, to create effects similar to the products offered in the below link :-

http://www.orad.tv/en/page.asp?id=85

Answer 1

Please look into OpenCV in all its incarnations.

Answer 2

If you don't object to having markers in your video. This artoolkit (Augmented reality tookit) might be just for you

Answer 3

Essentially you need a library that will decode a video file and present you with the raw data for you to analyse. There are no libraries (certainly none that I am aware of anyway) that will allow you to do the high level tracking directly (i.e. there won't be a findBall() method!!!). You will almost certainly have to gain access to the pixel values and analyse them yourself.

For this kind of thing, I would use something like opencv. It's a C++ library that will (1) allow you to gain access to the raw pixel data (as well as motion vector data etc), (2) provide you with some utility methods for analysis, and (3) allow you to overlay graphics on top of a video frame.

Overlaying the graphics on the video is the easy part, automatically tracking the players and detecting the situations etc is a lot harder. Usually it involves a mixture of analysis defining what you are looking for, and some machine learning algorithm to detect the specific events.

[edit]:

For what it's worth, I believe a lot of soccer analysis techniques use the lines on the pitch as markers for analysis, so a first step would be to detect them. The Hough Transform is a good way to do this (openCV will generate it for you). The next step would be to look for moving blobs (players) of a similar colour and size within these lines (i.e. ignore the crowd and ballboys etc). You can then use a combination of motion vector tracking and colour tracking to keep track of where they are. This will be a little difficult, since you have a moving camera and also moving players. Tracking the football can be done by essentially looking for a fast-moving small blob.

I would imagine (guessing here) that the makers of that software trained their system using some form of classifier to find free kicks etc, this will essentially say: "Find me situations when there is a stationary small white blob (the ball), with some larget blobs of a similar colour nearby (the players taking the free kick), and a large number of different colour blobs near the right/left extremety of the pitch (the players beside the goal)".

[/edit]