An iPhone library for shape recognition via the camera.

Question

Hi

I hope this falls within the "programming question" category. Im all lightheaded from Googling (and reading every post in here on the subject) on the subject "Computer Vision", but Im getting more confused than enlightened.

I have 6 abstract shapes printed on a piece of paper and I would like to have the camera on the iPhone identify these shapes (from different angles, lightning etc.).

I have used OpenCV a while back(Java) and I looked at other libraries out there. The caveat is that it seems that either they rely on a jail broken iPhone or they are so experimental and hard to use that I would probably end up using days learning libraries only to figure out they didn't work.

I have thought of taking +1000 images of my shapes and training a Haar filter. But again if there is anything out there that is a bit easier to work with I would really appreciate the advise, suggestion of people with a bit of experience.

Thank you for any suggestion or pieces of advise you might have:)

Answer 1

While not a library, Chris Greening explains how iPhone Sudoku Grab does its image recognition of puzzles in his post here. He does seem to recommend OpenCV, and not just for jailbroken devices.

Also Glen Low talks a bit about how Instaviz does its shape recognition in an interview for the Mobile Orchard podcast.

Answer 2

I do shape recognition in my iPhone app Instaviz and the routines are actually packaged into a library I call "Recog". Only problem is that it is meant for finger or mouse gesture recognition rather than image recognition. You pass the routines a set of points representing the gesture and it tells you whether it's a square, circle etc.

I haven't yet decided on a licensing model but probably use a minimal per-seat royalty.

Answer 3

Have a look at at OpenCV's SURF feature extraction (they also have a demo which uses it to detect objects). Surf features are salient image features which are invariant to rotation and scale. Many algorithms detect objects by extracting such features from an image, and then use simple "bag of words" classification (comparing the set of extracted image features to the features of your "shapes". Even without referring to their spacial alignment you can have good detection rates if you only have 6 shapes).