application of AI/neural networks/machine learning in stock market trading: looking for a book(s)

Question

Hello.

I'm looking for a book(s) about practical application of machine learning, artificial intelligence and neural networks to stock market trading (automated trading or as an assistance to human, mostly automatic trading). I'm not afraid of "heavy reading".

What I'm interested in:
0. How can the problem (how to achieve goal depending on variation of seemingly chaotic parameter) be approached - which theories are there, algorithms, etc.
1. What systems were already created, what they used (language/algorithm/theory/hardware), how successful they were?

Can you recommend something?

Answer 1

Can't help you with the specifics, but I've heard economists tend to think of machine learning as just another form of regression. So you might be successful in your search if you add a few more classical keywords...

Answer 2

There are a six set of books all directed to Quantitative Finance and published under a single Project, RMetrics. I have only read the sample (free) chapters from Basic R for Finance, though based on that limited exposure i can certainly recommend it.

Of these six, the one most responsive to your question is probably Portfolio Optimization with R/RMetrics. Revolution Analytics published a review of this book shortly after it was released.

All of these are available as e-books (which can be purchased from the page i linked to above).

Again, these books are all based upon RMetrics, a Project conceived by Diethelm Würtz and his group at the Econophysics Group at the Institute of Theoretical Physics of ETH Zurich.

So RMetrics is not actually a single R library or package but rather a single rubric which subsumes quite a few individual packages (about 30 i think, most of then have names that begin with 'f' e.g., fAssets (Asset Selection and Modelling), fNonLinear (Nonlinear and Chaotic Time Series Modeling), some are utility packages (fCalendar), some have only financial data sets (fEcofin), while others contain the predictive algorithms (fRegression, fAssets).

I can't actually tell you all of the RMetrics packages that have ML algorithms in them although i am certain that fNonlinear does (neural network).

Like all R Packages, these are all available on CRAN; however, if you suspect you want all of them, you can download and install the entire set (all of RMetrics) from the RMetrics Wiki.

It might be obvious at this point, but these packages run under R, the free, open-source statistics platform (also downloadable from the link to CRAN above, the R package repository).

Of the RMetrics, packages, i have only used RBasic; i do however use R every day and my sense is that it would be an excellent platform for the Quantitative Finance domain.

Answer 3

Amazon has quite a bit of reading on the topic, but I haven't really read any of those books, so I can't give you a useful suggestion on a specific book.

When it comes to Machine Learning and Artificial Intelligence, there are MANY algorithms out there, but they need substantial tweaking before you actually see some promising results. One thing that I've noticed is that whatever algorithm you use, you need to do several basic things:

1. Split your data set in two sets: training and validation.
2. Train your algorithm on the training data set and only use the validation data set to gauge how your best "strategies" are performing on a data set that they have never been exposed to before.
3. Do not, under any circumstance, use the validation data set to provide feedback to your "strategies".
4. Back test your strategies in an environment that is as realistic as possible.

All of those things might seem like no-brainers, but they make quite a bit of difference!

Stock-Forecasting is an interesting web site that uses neural networks to make stock market predictions. They show all of their predictions and you can even use some of their demo stocks to evaluate their profitability.

The participants in the Automated Trading Championship have shown some impressive results, although they don't share their solution with the public.

The programming language that you chose should be something that you're comfortable with... I would recommend that you stick with the classic C++, Java or C#, because a solution in any of those 3 languages will be easily plugable into the majority of trading APIs out there.

If you want to have a realistic trading simulation environment, then I would try NinjaTrader, T4 API, IB API (it's a bit cumbersome), or you could just download Yahoo Historical prices and run the simulations yourself.

Update

I just want to make a side note here as to why some of these answers might be vague: there are very few people that will actually openly discuss the details of their profit-making algorithms. Given that situation, all I can say is that the Automated Trading Championship (ATC) provides about the most reliable evidence that ML/AI algorithms can be used to successfully trade (and make profits). Note that the ATC a real-time championship, meaning that the participants place real-time orders that get filled at real-time prices from the exchange... the championship runs for 3 months and the 2008 winner started with $10,000 and ended with over $150,000. The algorithms run on the computers of the championship organizers for the entire 3 months, and the participants have no way of modifying anything on their systems during that time.

The only downfall of the championship is that all the submitted algorithms must be in a custom programming language (similar to C++) that the organizers have developed themselves, so they can easily reverse engineer any solution that is submitted... which is a good deal when they're only paying $40,000 to the winner and they're getting solutions that can make them over $150k in 3 months.

ATC Winner's Comments

In an interview with the winner of the ATC, Kiril Kartunov (yes, we do have the same first name and yes we are from the same country), he explains some of the details of his Expert Adviser:

1. Uses simple indicators: in particular the volatility adjusted indicator/oscillator.
2. Utilizes Repeated Incremental Pruning to Produce Error Reduction (RIPPER) to produce decision trees which make the trading signals.
3. Does not use stop-loss or limit orders of any sort, because he relies on the algorithm to take him out of a bad position, rather than the market taking him out.

Read through his interview and you might find some interesting gems that could be useful for your future work.