Transforming captured co-ordinates into screen co-ordinates

Question

Hi everyone. I think this is probably a simple maths question but I have no idea what's going on right now.

I'm capturing the positions of "markers" on a webcam and I have a list of markers and their co-ordinates. Four of the markers are the outer corners of a work surface, and the fifth (green) marker is a widget. Like this:

Here's some example data:

• Top left marker (a=98, b=86)
• Top right marker (c=119, d=416)
• Bottom left marker (e=583, f=80)
• Bottom right marker (g=569, h=409)
• Widget marker (x=452, y=318)

I'd like to somehow transform the webcam's widget position into a co-ordinate to display on the screen, where top left is 0,0 not 98,86 and somehow take into account the warped angles from the webcam capture.

Where would I even begin? Any help appreciated

Answer 1

Since your input area isn't a rectangle of the same aspect-ratio as the screen, you'll have to apply some sort of transformation to do the mapping.

What I would do is take the proportions of where the inner point is with respect to the outer sides and map that to the same proportions of the screen.

To do this, calculate the amount of the free space above, below, to the left, and to the right of the inner point and use the ratio to find out where in the screen the point should be.

Once you have the measurements, place the inner point at:

x = left / (left + right)
y = above / (above + below)

This way, no matter how skewed the webcam frame is, you can still map to the full regular rectangle on the screen.

Answer 2

Try the following: split the original rectangle and this figure with 2 diagonals. Their crossing is (k, l). You have 4 distorted triangles (ab-cd-kl, cd-ef-kl, ef-gh-kl, gh-ab-kl) and the point xy is in one of them.

(4 triangles are better than 2, since the distortion doesn't depend on the diagonal chosen)

You need to find in which triangle point XY is. To do that you need only 2 checks:

1. Check if it's in ab-cd-ef. If true, go on with ab-cd-ef, (in your case it's not, so we proceed with cd-ef-gh).
2. We don't check cd-ef-gh, but already check a half of it: cd-gh-kl. The point is there. (Otherwise it would have been ef-gh-kl)

Here's an excellent algorythm to check if a point is in a polygon, using only it's points.

Now you need only to map the point to the original triangle cd-gh-kl. The point xy is a linear combination of the 3 points:

x = c * a1 + g * a2 + k * (1 - a1 - a2)
y = d * a1 + h * a2 + l * (1 - a1 - a2)
a1 + a2 <= 1

2 variables (a1, a2) with 2 equations. I guess you can derive the solution formulae on your own.

Then you just make a linear combinations of a1&a2 with the corresponding points' co-ordinates in the original rectangle. In this case with W (width) and H (height) it's

X = width * a1 + width * a2 + width / 2 * (1 - a1 - a2)
Y = 0 * a1 + height * a2 + height / 2 * (1 - a1 - a2)

Answer 3

Due to perspective effects linear or even bilinear transformations may not be accurate enough. Look at correct perspective mapping and more from google on this phrase, may be this is what you need...