How to measure distance to object given two images from two cameras set at a known distance apart

Question

I'm looking for an open source project or library (c++, c) that can give me a method to calculate the distance to a certain object (pixel) in the image, assuming I have all the info about the cameras - The field of view, focal length, distance between cameras, etc..

Answer 1

I don't think there is an off the shelf solution for your needs. 3 years ago I did something similar using openCV, but I wrote most of the code by myself. I don't know if, in the meantime, the openCV guys have implemented this kind of computations.

Answer 2

I'm looking for an open source project or library (c++, c) that can give me a method to calculate the distance to a certain object (pixel) in the image, assuming I have all the info about the cameras - The field of view, focal length, distance between cameras, etc..

Depending on what exactly "etc" is, the solution is somewhere between easy and impossible.

If you have fully calibrated cameras (intrinsic parameters + relative extrinsic parameters) computing the distance is a matter of computing and intersecting two lines of sight. The lines of sight directly follow from all the known parameters and the pixel positions. Intersecting them is a matter of solving a linear least squares problem with 4 equations and 3 unknowns.

If you only have intrinsic camera parameters (Up to five degrees of freedom, usually an upper triangular 3x3 matrix) and the distance between the cameras you need to first reconstruct the remaining 5 degrees of freedom for the relative pose of the two cameras. This can be done by computing the "fundamental matrix" F for a set of point correspondences and dividing it by the known kamera matrices so you get the "essential matrix" E. This matrix can be further factored into a rotation and a cross product matrix. This tells you how the second camera is positioned relative to the first. Combinind this with the known distance between cameras, you'll be able to compute a metric reconstruction of things "you see".

If you don't even know all the intrinsic parameters you can't compute a metric reconstruction but only some kind of "projective reconstruction". But that's not really what you want.

Suggested reading: Multiple View Geometry by Hartley and Zisserman