views:

95

answers:

2

I saw that someone has made an app that tracks your feet using the camera, so that you can kick a virtual football on your iPhone screen.

How could you do something like this? Does anyone know of any code examples or other information about using the iPhone camera for detecting objects and tracking them?

A: 

I have had some success tracking faces and eyes using OpenCV on the iPhone. Here's a good place to start: http://niw.at/articles/2009/03/14/using-opencv-on-iphone/en

I guess the trick is finding a cascade (description of what the camera should be looking for) that describes a foot, not really sure if that exists though.

aegzorz
Can OpenCV be used to track in realtime? The linked article seems to indicate that it takes up to 10 seconds to recognize a face in a 480 x 320 image.
Brad Larson
I've used the CoreVideo framework on an iPhone 3GS to track a face in realtime, using the small sample size AVCaptureSessionPresetLow. I was able to consistently detect eyes at under 4 ms.
aegzorz
That sounds very good, I don't suppose you want would have a sample?
Burf2000
+2  A: 

I just gave a talk at SecondConf where I demonstrated the use of the iPhone's camera to track a colored object using OpenGL ES 2.0 shaders. The post accompanying that talk, including my slides and sample code for all demos can be found here.

The sample application I wrote, whose code can be downloaded from here, is based on an example produced by Apple for demonstrating Core Image at WWDC 2007. That example is described in Chapter 27 of the GPU Gems 3 book.

The basic idea is that you can use custom GLSL shaders to process images from the iPhone camera in realtime, determining which pixels match a target color within a given threshold. Those pixels then have their normalized X,Y coordinates embedded in their red and green color components, while all other pixels are marked as black. The color of the whole frame is then averaged to obtain the centroid of the colored object, which you can track as it moves across the view of the camera.

While this doesn't address the case of tracking a more complex object like a foot, shaders like this should be able to be written that could pick out such a moving object.

Brad Larson
Wow, much respect! I'll be using this if you don't mind ,)
Eimantas