I have an image displayed on screen which is undistorted via cvInitUndistortMap & cvRemap (having done camera calibration), and the user clicks on a feature in the image. So I have the (u,v) pixel coordinates of the feature, and I also have the intrinsic matrix and the distortion matrix.
What I'm looking for is the equation of the 3D line in camera/real-world coordinates on which the feature the user clicked must lie. I already have the perpendicular distance between the camera's image plane and the feature, so I can combine that with the aforementioned equation to give me the (X,Y,Z) coordinate of the feature in space.
Sounds easy (inverse intrinsic matrix or something?) but I can't find step-by-step instructions anywhere. C++ or C# code preferred.