It really helps to have a clear head for figuring this stuff about. It is pointless to go straight to the math without a really clear picture of what you want to do. And this sort of stuff is guaranteed to get your brain in a twist, so you need to get the essentials.
Imagine you are sitting, with a closed book on your lap, with the sun setting over the sea ahead of you.
Now imagine you are told to hold the book out so that its front cover faces the setting sun. Now you would probably have the title text the right way up, but equally it could be upside down or sideways.
i.e. the task was not specified completely. The specification was too loose. And so you get not a unique solution but a solution set. A single degree of freedom.
To resolve into a unique solution, you need to give an additional constraint.
Lets say you supply the additional constraint that the writing be the correct way up.
Now the sun could be anywhere in the sky, and you will arrive at unique solution.
Anywhere, that is, apart from directly overhead/underneath. You would need to take this as a special case.
In fact two rotations would be sufficient.
Imagine you stick a long nail through the book so that its tip faces upwards. Let's say that is the Z axis.
Now your first rotation is about the axis of the nail. Imagine the book has a clock face printed on the cover, and the nail has come through the centre. Simply rotate until the shadow that the nail casts equals 12 o'clock on the clock face.
The second rotation is to lift the top of the book until the shadow has receded to a single point. this is actually rotating around the axis that is the bottom edge of your book. but you could equally well rotate around the line that joins the nine o'clock position to the three o'clock position. Let's say that is the X axis.
( Of course you would need to look carefully at the situation when the sun is beneath the horizon... maybe find a more apt analogy )
I wrote a ray tracer once, years back. And it consisted of tons of sticking pencils through the bits of paper, and chewing on abstract thought experiments.
Always start from a point you understand. Never be in a position where you're tweaking some magic formula that doesn't quite work. I got through a ton of A4 refill with this stuff. always start a fresh page if it gets murky.