Calculating 'up vector' from transformation matrix in 3D

Question

Hello!

I just came to strange problem with my project in 3D. Everyone knows algorythm of calculating LookAt vector, but it is not so easly to calculate "up" vector from transformation matrix (or at least maybe I simple missed something).

The problem is following:

"Up" vector is (0, 1, 0) for identity rotation matrix and rotate with matrix, but do not scale nor translate. If you have simple rotation matrix procedure is easy (multiply vector and matrix). BUT if matrix contains also translation and rotation (e.g. it was produced by multiplying several other matrices), this won't work, as vector would be translated and scaled.

My question is how to get this "up" vector from single transformation matrix, presuming vector (0, 1, 0) correspond to identity rotation matrix.

Answer 1

I'm no expert at matrix calculations, but it strikes me as a simple matter of calculating the up vector for the multiplied matrix and normalizing the resulting vector to a unit vector. Translation shouldn't affect it at all, and scaling is easily defeated by normalizing.

Answer 2

Simply multiply the up vector (0,1,0) with the transformation, and normalize. You'll get the new calculated up vector that way.

Answer 3

Apply your matrix to both endpoints of the up vector -- (0, 0, 0) and (0, 1, 0). Calculate the vector between those two points, and then scale it to get a unit vector. That should take care of the translation concern.

Answer 4

Translation actually does affect it. Let's say in the example the transformation matrix didn't do any scaling or rotation, but did translate it 2 units in the Z direction. Then when you transform (0,1,0) you get (0,1,2), and then normalizing it gives (0,1/sqrt(5), 2/sqrt(5)).

What you want to do is take the difference between the transformation of (0,1,0) and the transformation of (0,0,0), and then normalize the resulting vector. In the above example you would take (0,1,2) minus (0,0,2) (0,0,2 being the transformation of the zero vector) to get (0,1,0) as desired.