Mathematics for AI/Machine learning ?

Question

I intend to build a simple recommendation systems for fun.

I read a little on the net and figured being good at math would enable on to build a good recommendation system. My math skills are not good.

I am willing to put considerable efforts and time in learning maths. Can you please tell me what mathematics topics should I cover? Also if any of you folks can point me to some online material to learn from it would be great.

I am aware of MIT OCW, book like collective intelligence. Math Topics to cover and from where to read would really help.

Answer 1

One book you might want to look at is Programming Collective Intelligence by Toby Segaran. It's not too theoretical, directly applied to programming.

If you want to learn more theoretical background, I'd recommend learning the basics of Bayesian statistics. A lot of AI and machine learning consists of ad hoc approximations to Bayesian statistics.

Answer 2

For your specific project I suggest you read up on logic and probability. Under logic you'll want to understand Fuzzy Logic, and under probability you'll want to understand Bayes Theorem in particular. The wiki links are just a starting point. Another good online resource for learning about mathematical topics is Wolfram MathWorld.

The three general areas of mathematics that are most significant to the study of AI are logic, theory of computation, and probability and statistics. My book recommendations for these subjects are:

• Logic - An Investigation of the Laws of Thought (Boole) and Set Theory and Logic (Stoll)
• Computation - Introduction to the Theory of Computation (Sipser)
• Probablility - Any college level statistics book should have a section on probability. Pick one that your college uses, or one that you like.

You'll probably not run into the limits of the theory of computation for your specific project, but I still recommend the book by Sipser, as it is an excellent read.

One last book that I recommend to anyone studying in any branch of computer science is Concrete Mathematics. The authors give a really good overview of all the branches of math that you'd encounter in a good undergraduate CS program.

Addendum: Steve Yegge also gives a good listing of what math a programmer will need in his blog post Math for Programmers.

Answer 3

Hi, As usual the wikipedia entry on Artifial Intelligence is a good beginning, at least that entry will give you some topics for starters.

Answer 4

This woman Mikaela Keller seems to argue that probability,statistics and linear algebra are needed.
And altough her talk may not be one of the best,it gives you ideas about what you should
look closely at when learning.

Answer 5

From my experience, probability will be an important thing to know ahead of time. For those not used to it (like me) it can be very non-obvious. Just get a basic probability and statistics book and keep it as reference. If you're going to do any kind of statistical data analysis, like say a Bayesian network, you're going to be calculating a lot of probabilities. I also learned the hard way that I have to know how to do matrix multiplication, which I'd never seen. The logic aspects are important, too, but from a Math perspective, I found probability to be the killer.

Answer 6

Recommendation system - sounds like you will try to optimize something given "too much" information, similar to finding a line that passes close to a given set of points. (A line here represents the type of stuff the user are interested in, and the points are what the user had decided he/she is intrested in).

Computations of this type will require some linear algebra (matrixes and stuff), and algorithms to find clusters of similar items, http://en.wikipedia.org/wiki/Cluster_analysis

Answer 7

If you are going to do machine-learning/statistics, in addition to statistics (and maybe linear algebra), it is absolutely crucial that you understand what "Over-fitting" is and how to avoid it.

I'll explain over-fitting with an example.

• Let's say you trained a neural-network to learn to separate between cat pictures and dog pictures.
• Your training set consisted of 10 cat pictures and 10 dog pictures, and it has 10MB of uncompressed data overall
• You set the "model data" your neural-network results in after training to a size of 20MB
• Your neural network, after the training process, knows perfectly two identify if it's a cat or dog in the pictures in your training set
• If you give your model a picture you didn't train it with, it gives the correct answer 50% of the time. Like guessing at random.
• This happened because the model size is bigger than the training set's size and so may allow the model that says that knows the answers for the training-set but makes no generalization whatsoever and so have learned nothing about general pictures of cats and dogs.

There are several ways to battle over-fitting. A very common one is to limit yourself to compact models. This is kinda an application of "Occam's Razor" to your models.

If philosophy interests you, then over-fitting is closely related to "The Problem of Induction".