Let's say I have two points in 3D space (a and b) and a fixed axis/unit vector called n.
I want to create a rotation matrix that minimizes the euclidan distance between point a (unrotated) and the rotated point b.
E.g:
Q := matrix_from_axis_and_angle (n, alpha);
find the unknown alpha that minimizes sqrt(|a - b*Q|)
Btw - If a solution/algorithm can be easier expressed with unit-quaternions go ahead and use them. I just used matrices to formulate my question because they're more widely used.
Oh - I know there are some degenerated cases ( a or b lying exactly in line with n ect.) These can be ignored. I'm just looking for the case where a single solution can be calculated.