As said above, the best book you can possibly get is Physically based Rendering by Matt Pharr (check out www.pbrt.org). Explains a lot of algorithms in great detail, including advanced stuff like photon mapping. Moreover, it includes a fully working ray-tracer, so you can take a look at it. It also covers the math basics, so if you don't want to buy many books, I'd definitely recommend to take a look at this one.
It's much better than the classic books on this subject, as they tend to explain only the theory, not so much how to really implement it. For basics, any math book will do it, or you could try "Realtime collision detection", which also explains lots of intersection routines (which you will need in ray-tracing).
If you really want to start at the basics, you should try "Computer Graphics: Principles and Practice", it's dated (some parts are really nonsense now), but it explains the basics pretty well. If you want a more recent book, try "Fundamentals of Computer Graphics", which contains the same, just not that detailed (should be good enough to get you started thou).
Last but not least, the wikipedia page on Raytracing is actually pretty good and should give you some starting points. Take a look at the external links section.