Thinking in regards to Sliverlight, I would like to know where would be good places to go to get a refresher on 3d space, transforms, matrix manipulation, and all that good stuff.
Think I may have found it myself. Was looking at: http://msdn.microsoft.com/en-us/library/cc189037(VS.95).aspx and http://www.c-sharpcorner.com/UploadFile/mgold/TransformswithGDIplus09142005064919AM/TransformswithGDIplus.aspx
Any linear algebra textbook should provide the math refresher; there's a fairly good one available online at Linear Algebra textbook home page.
It's not a place, but I've found 3D Programming for Windows by Charles Petzold excellent. It covers everything you ask about and is focused specifically on WPF/silverlight.
Of course Petzold (as usual) is able to communicate the important concepts beautifully.
Free graphics algorithms can be found in the comp.graphics.algorithms faq
As previously mentioned you should really learn linear algebra, here are some great video lectures about it, MIT Linear Alebgra Video Lectures.
Personally I think that although the bible (by Foley & Van Damn that is) was the greatest book for its time, but it is somewhat outdated.
I would suggest 'Advanced animation & Rendering techniques' by Alan and Mark Watt. The only problem with this book is that it gives you a good understanding almost about every broad aspect in CG but it assumes you have some familiarity with it, and does not explain it all the way. You can always have a look in the Bib and find enhanced articles and books about each subject you are interested in depth.
If you want further on once you have more understanding, or if you want to dive into the world of computer graphics and the use of GPU I suggest to have a look at the three 'GPU Gems'.