OpenCV / SURF How to generate a image hash / fingerprint / signature out of the descriptors?

+2 A:

The feature data you mention (position, laplacian, size, orientation, hessian) is insufficient for your purpose (these are actually the less relevant parts of the descriptor if you want to do matching). The data you want to look at are the "descriptors" (the 4th argument):

void cvExtractSURF(const CvArr* image, const CvArr* mask, CvSeq** keypoints, CvSeq** descriptors, CvMemStorage* storage, CvSURFParams params)

These are 128 or 64 (depending on params) vectors which contain the "fingerprints" of the specific feature (each image will contain a variable amount of such vectors). If you get the latest version of Opencv they have a sample named find_obj.cpp which shows you how it is used for matching

update:

you might find this discussion helpful

liza 2010-01-27 19:52:20

Hi Liza, thanks for your reply. I looked at the example but don't find any way to transform the vectors of the descriptors stored in CvSeq into an hash value that makes it possible to be stored in a database and be compared to other hashes. Do you have any hint on that?

dan 2010-01-27 21:48:56

the CvSeq simply stores float arrays (look at the source code of cvExtractSURF to see how they store it). those arrays (128 or 64 in length) are what you are interested in

liza 2010-01-27 23:25:35

by the way, if what you want is get some magical hash which would map all dog images for example to the same hash code, i'm afraid it is more complicated than just extracting the SURF features. Have a look at the link I put on the update

liza 2010-01-27 23:28:16

No, I'm trying to find the same picture within a photo. The OpenCV Sample works great with all of my test shots.But I'm still having no clue how to transform the descriptors into a linear hash value to use the Hamming distance to calculate the similarity.

dan 2010-01-28 15:48:55

You need to define what you mean by "linear hash" and what you require of it. If I understand you correctly what you really want is a similarity measure between two image, is this the case?

liza 2010-01-28 16:20:52

The OpenCV example simply uses the euclidean distance between the descriptor vectors in order to fetch the "nearest neighbor". That means the matching of features is performed by taking a feature vector, computing the (euclidean)distance from it to ALL(hence the "naive") other features and choosing the one with the minimal distance.

liza 2010-01-28 16:27:53

Liza, thank you very much for all your help.You are right, I'm looking for a hash fingerprint as a numeric string for every image that can be stored in a relational database. Than I want to calculate the Hamming distance between two images to check if the images are almost the same. My problem is that I don't know how to convert the descriptors vector data into such an string - just like the pHash library does.

dan 2010-01-28 19:27:31

I've added an update to my questing clarifying what I've meant by linear hash.

dan 2010-01-28 19:44:30

+1 A:

A trivial way to compute a hash would be the following. Get all the descriptors from the image (say, N of them). Each descriptor is a vector of 128 numbers (you can convert them to be integers between 0 and 255). So you have a set of N*128 integers. Just write them one after another into a string and use that as a hash value. If you want the hash values to be small, I believe there are ways to compute hash functions of strings, so convert descriptors to string and then use the hash value of that string.

That might work if you want to find exact duplicates. But it seems (since you talk about scale, rotation, etc) you want to just find "similar" images. In that case, using a hash is probably not a good way to go. You probably use some interest point detector to find points at which to compute SURF descriptors. Imagine that it will return the same set of points, but in different order. Suddenly your hash value will be very different, even if the images and descriptors are the same.

So, if I had to find similar images reliably, I'd use a different approach. For example, I could vector-quantize the SURF descriptors, build histograms of vector-quantized values, and use histogram intersection for matching. Do you really absolutely have to use hash functions (maybe for efficiency), or do you just want to use whatever to find similar images?

2010-01-30 01:22:19

Hi Sheldon,thank you for your reply.Before looking at SURF I was working with the pHash lib. That lib generates a hash value for media data (images, video and audio). Than you could use the Hamming distance to query your database of similar image hashes quite easily.I thought there must be a way to do the same with the SURF descriptors in some way.Now I'm thinking of storing the all the serialized image descriptor array-objects in the database. When I want to find a similar image in the database I have to select row by row and finding similar descriptors by hand. (...)

dan 2010-01-30 16:25:42

(continued)The big downside would be a very bad performance for a large image database.I'm wondering how TinEye.com could find similar images so quickly. There must be some trick to query their database very fast.

dan 2010-01-30 16:28:53

I think what would help is if you told what exactly are you trying to achieve. Given an image, what are you trying to find? Identical images, similar images, similar images up to rotation/translation scaling? Given an image X, do you want to find only images that are similar to X, or do you also want images that contain X as a sub-part? For example, the query is a face image. Do you want to find other face images, or also full-body photographs that contain the face but also a lot of other stuff? Why did you stop using pHash and decided to work with SURF descriptors instead?

2010-01-31 01:43:00

What I'm trying to achieve is to build up a database containing signatures of sample images. Then I want to print out those sample images and take pictures with them with my mobile phone. Then I'm trying to match the photos with the original images from the database. Since the photos will have a different lighting and could be slightly distorted pHash will not work because this lib is not tolerant enough to lightning and other error sources. The SURF algorithm works great. The only problem I have is to store the descriptors in the database in an efficient way for fast querying.

dan 2010-01-31 03:00:25

A:

i use cvsurf descriptor like you and i try to do the same thing as you but i dont know how

what exactly from code must convert from vector to integer ?

and i have a question i want to find out how many degrees are different object in relation to another (each object located in different images)

thanks

ntouros 2010-02-05 18:20:13

A:

It seems like GIST might be a more appropriate thing to use.

http://people.csail.mit.edu/torralba/code/spatialenvelope/ has MATLAB code.

forefinger 2010-02-08 20:24:36

A:

Dan did you find a solution for your question? This is very interesting question how to store image fingerprints in database for further search..

Kirill 2010-03-02 22:00:25

A:

Hi there,

does anyone heard of a VQ implementation in C or C++ ? I'd like to vector quantize Surf descriptors but don't know how to practically do it.

Thx

Jayka 2010-07-26 14:12:19

ansaurus

tags:

views:

answers:

OpenCV / SURF How to generate a image hash / fingerprint / signature out of the descriptors?

related questions