Rectify perspective image of planar structure given the normal vector of the plane and viewing direction of the camera

Question

Hi,

I have taken images of the facade of a building under multiple view points. Then I use bundler to reconstruct the 3D point cloud of the building. Then I use a RANSAC method to find the plane of the facade in the point cloud.

This gives me the parameters for the plane and its surface normal.

From bundler I also get the camera rotation matrices, which I use to calculate the viewing directions of the cameras.

Now given that I have the normal of the plane and the camera viewing directions, I would like to use this to rectify the images, i.e. as if the images where taken with the camera's viewing direction being the same as the normal of the plane.

How can I calculate the projective transformation matrix from these two vectors?