what's a good language for learning machine learning?

Question

I've been thrust into a situation where I need to know something about Machine Learning. Is there a language or perhaps a reasonable tutorial that breaks this subject matter in gently? I'm not a math guy, so it's got to start from a pretty basic level.

Answer 1

Here is an interesting article with some good recomendations and tips for getting started. http://hunch.net/?p=230

Answer 2

I'd say start with the Math then. I'm not sure what your time line is, but if you really want to get it. Understand the Mathematics of it, and then learn some implementations.

A decent online resource to get you started.

Answer 3

You're going to have to be more specific about what aspect of Machine Learning you want to focus on. Is this Neural Networks / fuzzy logic, or is this going to be an Expert System?

Answer 4

Machine learning is such a broad subject that it really depends on what type of learning you're intending to achive as to what you should be understanding.

For example, do you intend to have your machine learn by use of a Neural Network, or by a set of algorithms designed to alter states programatically?

It's very hard to give a broad overview of this subject!

Answer 5

Start with C. It has been termed as "portable machine code". If you can use GCC, then write simple C programs and compile them with the -S flag to see what the generated assembly code is.

I don't know what system you are on but get the official manuals for your CPU (I know Intel's a free to download and "easy" to read) and read the index, for any op-code that you can't guess what they do, skim the section. If you are comfortable with reading and following GCC's output and have some idea what all the op-code do you are well on your way.

Just keep in mind that ASM has no safety net, it does exactly and only what you tell it to.

Answer 6

It may be a bit overkill for a tutorial, but Stanford released one of their courses on machine learning a while ago to the internet. Here

For the purposes of learning machine learning, I used Python to implement a bunch of the algorithms when I took a course in it at University, and aside from the performance issue, I don't regret it.

You might also look into R language, which is built for statistical computing.

Answer 7

The AI languages of choice for many years have been Prolog and Lisp. You should definitely have a look at these, even if they aren't considered 'modern'.

Don't worry about maths, most machine learning is logic-based.

Answer 8

A fellow in the research group I used to work in at the University of Illinois was doing work on learning-based programming, that is, programming languages where machine learning concepts had native support as language constructs. The language he was implementing ultimately compiled into Java that was compiled by the normal Java compiler so you could call it from Java.

Other than that, we had a few tools that were written in C++ that you could re-use, if you liked. It would probably be fairly easy to write wrappers to call these from other languages. I know they would appreciate the contribution.

Answer 9

If you know Java, maybe you should have a look at weka, it's a pretty nice, extensible open source toolkit with a large mailing list and lots of documentation. I can also recommend the corresponding book, which discusses the most important general issues (prediction error estimation etc.) and walks you through several fundamental algorithms. Most math is explained really well and is usually detailed in extra sections marked as technical, so that you can read around it. Still, you will need to understand some key mathematical concepts (e.g. approximation, entropy) if you want to understand why the algorithms work.

Answer 10

For a good introduction to how machine learning ideas can be implemented in Python (including decision trees, neural networks, genetic algorithms, clustering, etc) I would point you at the O'Reilly book "Programming Collective Intelligence" (ISBN 10: 0-596-52932-5 | ISBN 13: 9780596529321 - sorry I'm a new user so I'm not allowed to post hyperlinks). Python is a very suitable language for writing many machine learning type programs but apart from the language itself there are also many additional libraries that you can download and use (e.g. LIBSVM for classification using support vector machines). Python is an interpreted language so, depending on what you're doing, you may have an issue with performance but often the third party libraries provide efficient implementations (e.g. of number crunching operations).

Answer 11

I would say it depends on what will be your tasks. If it is to:

• design and test clustering algorithms, take a look at Weka using Java or PyBrain and SciPy using Python.
• implement fast clustering based on support vector machines (SVM) or artificial neural networks (ANN) both using C++.

Anyway, if you have to deal with a lot of statistics, you should take a look at R.

Pointers:

• As said above, tutorials and online courses from Stanford / MIT and so on...
• For a Bayesian (quite complete) overview.
• search engine for ML.

Answer 12

I second python, there is a New book "Machine Learning An Algorithmic Perspective". There are lots of Python code examples in the book, and the code is available http://www-ist.massey.ac.nz/smarsland/MLbook.html .

Answer 13

I do most of my machine learning work in MATLAB, and some textbooks (like Bishop's latest one) provide MATLAB source for download somewhere. It has some nice toolboxes and stuff, but it all costs money. I'd probably work in Python/SciPy if my boss wasn't expecting MATLAB code.

Answer 14

I second Matlab.

Most researchers in ML area use Matlab, or partially use Matlab. A lot of researchers public their code in Matlab/Octave with critical part written in C/C++ for performance's sake.

This is only for machine learning.

For data mining, I think Python and Java are both quite good not because they do better in writing the algorithms, but because they are general purpose languages and easy to be used to preprocess various data, do some good reporting, etc.

I'd also like to mention F#, the newly released functional programming language by Microsoft. I wrote an article Why F# is the language for data mining. You may be interested in reading it.

Answer 15

Scala the Scala DSL OptiML is worth following