How to recognizing money bills in Images?

Question

I'm having some images, of euro money bills. The bills are completely within the image and are mostly flat (e.g. little deformation) and perspective skew is small (e.g. image quite taken from above the bill).

Now I'm no expert in image recognition. I'd like to achieve the following:

• Find the boundingbox for the money bill (so I can "cut out" the bill from the noise in the rest of the image
• Figure out the orientation.

I think of these two steps as pre-processing, but maybe one can do the following steps without the above two. So with that I want to read:

• The bills serial-number.
• The bills face value.

I assume this should be quite possible to do with OpenCV. I'm just not sure how to approach it right. Would I pick a FaceDetector like approach or houghs or a contour detector on an edge detector?

I'd be thankful for any further hints for reading material as well.

Answer 1

There is a good book on openCV

Using a Hough transform to find the rectangular bill shape (and angle) and then find rectangles/circles within it should be quick and easy

For more complex searching, something like a Haar classifier - if you needed to find odd corners of bills in an image?

Answer 2

Hough is great but it can be a little expensive

This may work:

-Use Threshold or Canny to find the edges of the image.

-Then cvFindContours to identify the contours, then try to detect rectangles. Check the squares.c example in opencv distribution. It basically checks that the polygon approximation of a contour has 4 points and the average angle betweeen those points is close to 90 degrees. Here is a code snippet from the squares.py example (is the same but in python :P ).

  ..some pre-processing
  cvThreshold( tgray, gray, (l+1)*255/N, 255, CV_THRESH_BINARY );

        # find contours and store them all as a list
        count, contours = cvFindContours(gray, storage)

        if not contours:
            continue

        # test each contour
        for contour in contours.hrange():
            # approximate contour with accuracy proportional
            # to the contour perimeter
            result = cvApproxPoly( contour, sizeof(CvContour), storage,
                CV_POLY_APPROX_DP, cvContourPerimeter(contour)*0.02, 0 );
            res_arr = result.asarray(CvPoint)
            # square contours should have 4 vertices after approximation
            # relatively large area (to filter out noisy contours)
            # and be convex.
            # Note: absolute value of an area is used because
            # area may be positive or negative - in accordance with the
            # contour orientation
            if( result.total == 4 and 
                abs(cvContourArea(result)) > 1000 and 
                cvCheckContourConvexity(result) ):
                s = 0;
                for i in range(4):
                    # find minimum angle between joint
                    # edges (maximum of cosine)
                    t = abs(angle( res_arr[i], res_arr[i-2], res_arr[i-1]))
                    if s<t:
                        s=t
                # if cosines of all angles are small
                # (all angles are ~90 degree) then write quandrange
                # vertices to resultant sequence
                if( s < 0.3 ):
                    for i in range(4):
                        squares.append( res_arr[i] )

-Using MinAreaRect2 (Finds circumscribed rectangle of minimal area for given 2D point set), get the bounding box of the rectangles. Using the bounding box points you can easily calculate the angle.

you can also find the C version squares.c under samples/c/ in your opencv dir.

Answer 3

You can also take a look at the Template Matching methods in OpenCV; another option would be to use SURF features. They let you search for symbols & numbers in size, angle etc. invariantly.