Maths in Programming - The Developer's guide

Question

Possible Duplicate:
Useful math for programmers

I'm a 22 year old ASP.NET/C#/Front-End developer with roughly 5.5 years experience. Unlike some (well probably most) of you, I didn't go to University to do a CS degree (*sigh*). The reasons behind that are, well... complicated, so I ended up teaching myself by reading books, articles, reading and posting on forums, experimenting, and so forth.

I always like to think I have a quite logical, problem-solving mind and it has definitely helped me over the years. I also like to think it's quite genetic since my dad was a very smart mathematician. Unfortunately I never really picked up much from him, but that's a different story.

After reading this question, I have to say that I agree with Nils' answer and that a developer should first know the fundamentals of programming (statistics, computing, databases, operating systems, computer architecture, computer networks and discrete mathematics) to name a few.

I have already considered going to university, but I could probably not afford to do it full-time, and doing it part-time would mean working 8-5, then studying 5-9. Its also likely that it would affect my work performance.

But anyway, enough about me – let's get to the question. :)

Based on the above, what resources would you suggest that are specifically related to Mathematics, especially the kind that is often used in programming? (statistics, algorithms, etc.)

I'm very interested in learning, and have mostly been inspired by you, the Stack Overflow community.

All answers will be considered.

Update

Wow, some excellent answers have come in and I'd like to thank everyone who has helped. I'm in the process of reading through all of them and seeing what the best place to start is. I am particularly interested in Boolean Algebra, Big-O notation, Calculus and basic Geometry to start with. One can only learn so much at once, so after getting myself more familiar with the topics above, I will likely move onto more complicated topics, especially since I will know exactly what I want/need to learn.

So once again, thanks to everyone for their great help, I will leave the question open in case others come with more information and I'll let the community vote up the best answers.

Answer 1

Big/Little O notation

Discrete Mathematics

Boolean Algebra

Modular Arithmetic

Of course there are many others but these would be the ones that apply to most situations and programming languages.

Lastly if you wish to get some practice at more mathematically oriented programming I recommend projecteuler

Answer 2

Based on the above, what resources would you suggest that are specifically related to Mathematics, especially the kind that is often used in programming? (statistics, algorithms, etc.).

I haven't used much maths in programming beyond big-O notation, boolean logic, and arithmetic: not because I didn't know any (my university degreee was in maths) but because the programming that people have asked of me didn't need it.

Only very occasionally have I used other maths-related things: huffman encoding, queueing theory, and a bit of signal analysis (for a signal-processing application).

It's not the programming that requires maths: it might be the problem domain (e.g., I don't know, maybe you might need to know some Financial mathematics if you were programming 'quant'-type software; or if you're doing signal processing, or statistics, or combinatorics).

So, to not answer your question, I think it's the other things that are more important: for example operating systems, databases, and networks.

Answer 3

I'm self educated in mathematics so I can talk about what helped me. I failed high school algebra twice (apparently, my strict no-homework policy didn't vibe to well with a school district that counts homework for at least 40% of the class) before I dropped out.

I should point out that while my list is not specifically geared for programming in any sense, it is all good to know and relates to it. It gives you the "I can understand anything" feeling. Want to know what the History Monoid is (relating to concurrent systems)? It's trivial if you know what a monoid and free monoid are. If not, you're screwed without a lot of hard work (after which, you'll know what a monoid and free monoid are ;)

I recommend that you learn linear algebra first. In saying that, I mean real mathematics, not the stuff that's focused on solving linear equations for practical applications but proof oriented mathematics. The only book I know of that approaches it from a beginners perspective and with the right attitude is Linear Algebra: An Introduction to Abstract Mathematics. It's a very fun book and reads more like a modern algebra text then the sort of garbage introductions that focus on purely numerical concerns that have flooded the market place. They all have such fugly notation too. Don't ever trust a math book with glossy pages and pictures of people skiing. If it's yellow though, it's good.

after that, I would recommend calculus. My recommended introduction is Essential Calculus with Applications. It's a little dover book (one of the good ones, they're hit or miss) and does a very good job of getting you up to speed on the concepts involved, e.g. it doesn't wave it's hands over what a limit is.

After that, check Vector Calculus, Linear Algebra and Differential Forms. It is a very good book that is solid on the theory and deals with actual computations. It is the sole exception to the 'glossy paged math books suck' rule that I have ever seen. No pictures of people skiing though and the authors (husband and wife) actually explain what the uber-trippy picture on the cover is and how it's calculated.

Then to continue the calculus (getting optional at this point) I recommend Undergraduate Analysis by Lang along with a schaums outlines (calculus and multivariate calculus) for plenty of concrete examples and techniques that Lang will assume known but which my earlier recommendations might skimp on.

After that you probably want to finish up the algebra. I learned my undergraduate level stuff from Introduction to Algebra. This is the only book I've seen that actually explains that a vector space is just an embedding of a field in the endomorphism ring of an abelian group. So now you don't need to remember all of those axioms any more, they fall right out of that definition.

Then you'll have plenty of experience writing proofs and be comfortable with groups and point-set topology. You could either dig into some graph theory taking things in the discrete mathematics direction (I'm still looking for a good book on it) or you could (if you "fell into the math hole") continue with Introduction to Topological Manifolds

If you're trying to learn math on your own, it's hard. I'd recommend that you find the Go club (game not language) near where you are because that's where a lot of mathematicians hang out and go is a great game to learn. Most of them are happy to help with problems if they see that you're working on it. When I was hitch hiking around out west one time, I was stuck on a problem and found the go club. When I got there, there were only two people and they were in a casual game so it was polite to talk. I asked them if either of them was a mathematician and they looked at me and exclaimed "We both are!". One of them was able to help me.

Also highly recommended after you finish the calculus stuff is Mathematics for 3D Game Programming and Computer Graphics. This is the book with the example that helped me finally grok co-variant vs. contra-variant tensors. It didn't use that terminology but I happened to grok them enough to spot the example and it all fell into place. The point is that it's a very good book that leans much more toward the intuition than the formalism.

Answer 4

That's an interesting question; and I would say that applies to most of us programmers after being the industry long enough - a refresher of the basics of math and stats is really helpful.

I found the the MIT algo course very useful and keep referring to it now and then: http://www.youtube.com/watch?v=JPyuH4qXLZ0

In general, courses on discrete math and fundamentals of statistics. I found this lecture series for stats - The UC Berkley Statistics 20 - but haven't come across anything good for discrete math though...

Answer 5

The notes on this introductory MIT course are extremely complete, it is almost like a textbook.

For a bit of history, there is this book called 'The Essential Turing'. It contains a number of papers from one of the founders of computer science (and a great mathematician). The papers are actually surprisingly accessible.

Answer 6

There's mathematical knowledge and then there's mathematics as a mean to develop abstract thinking. Topics like boolean algebra, mod operator, rounding errors are all useful knowledge but they can easily be learnt from the web. Abstract thinking and problem solving skills on the other hand requires lots of mind training. Very few software engineers ever use most of the physics or mathematics subjects they learnt at uni but the rigorous solving skills acquired is what counts.

Answer 7

There's a lot of mathematics that's specific to certain areas within software, but the more generally useful areas that I've come across are:

• logic and predicate logic: provides good understanding of basic operators, truth tables, and logical axioms. More thorough than straight boolean algebra because it also includes the concepts of 'any/all' or 'exists/not exists'.
• set, rings and fields: includes abstract structures, the concepts of isomorphism & polymorphism, inheritance. This also forms the basis for most cryptography work, in conjunction with large prime numbers, and some good set logic (relates to SQL)
• sums of infinite series / calculus, which leads to big O notation and algorithmic complexity.

I believe any sort of abstract mathematics is going to be useful, because it teaches you to think about abstract concepts - which is what the fun and challenging part of software engineering is all about.

I have a mathematics major instead of a CS major. I missed out on some useful knowledge but I believe I gained a lot more than I lost.

Answer 8

From a pro-mathematician: At least learn some combinatorics and linear algebra. I find the former helps me to express problems efficiently and organise my thoughts. It can also help to find an effective way to solve the problem.

To be honest anything that disciplines you to think and argue will do just as well. Even the ability to count is an underrated but essential skill.

Answer 9

There are many source and many software which will help you get more knowledge about mathematics. There are many Open Universities which were providing lots of distance learning course to understand and find out a certificate for yourself. Like In India - IGNOU which is one of best university to start your career too.

If You will do a mathematics course you can finish it simultaneously you can do your work without any loss.

You will get study materials and the process to start your career and many more new things too.

You will get practicals/projects and case studies and many more come up with it.

So after finishing up you can start your career for doctorate too.

I think It ll be better go for a distance course which will make a practice of learning something with a certification and Degree too.

Thanks for going through it. If any comments please add without any hesitation