In order of importance, I'd say:
- Algebra
- Combinatorics
- Statistics & Probability
- Vector Math
- Matrix Math
- Calculus
In my opinion, Algebra is essential for any decent programmer, since it forms the underpinnings of variable representation and expression evaluation (functions, commutativity and transitivity of operations, etc).
Combinatorics is essential for all sorts of graph algorithms (traversing all nodes in a graph, finding shortest paths, etc). Knowing your way around combinations and permutations is vital in almost any programming context.
Next in line is statistics, especially if you want to get into AI or machine learning. Stats is probably the biggest bang-for-your-buck area of math knowledge for programmers. In most cases, you don't need to get into the heavy stuff, like multivariate regressions. Just knowing how (and in what circumstances) to quickly calculate mean, median, mode, standard deviation, margin of error, confidence interval, and conditional probabilities is huuugely useful for lots of machine learning applications.
Vector math is important in a variety of information-modeling applications (n-dimensional vector-space models are a very handy way of reasoning about document semantics), as well as for all sorts of 3D graphics applications.
Matrix math is a lot like vector math, in terms of its applications (AI, information models, 3D, etc) but it's a little bit trickier, since it adds another dimension. (Matrix math is my own achilles heel; I could definitely use a brush-up.)
And calculus is essential for anyone doing scientific computing and simulation.
Have fun!
ON EDIT:
I think the best way to brush up on your math knowledge is to work on a project that uses that knowledge. If, for example, you want to learn some statistics (like, REALLY learn it, so that you can rattle off all the formulas without looking them up), then you might be interested in analyzing the dataset from the Netflix Prize contest.
The dataset includes more than 100 million records from the Netflix user database and offers an enormous opportunity for trying your chops at implementing some of those math-heavy algorithms.
As for actual reference materials, I've found the wikipedia to be more than adequate. For example, on a recent project of mine, I've repeatedly referred to some of its articles on Kernel Density Estimation and Vector Fields.
The intimidating thing (for me) about hardcore math (since I'm almost entirely self-taught) is the notation. All those greek letters put my head in a bit of a whirl. But if you read the articles several times over (and google for phrases like "math sigma symbol"), then it's not too difficult to piece together what they're talking about.