Building a Texas Hold'em playing AI..from scratch.

Question

I'm interested in building a Texas Hold 'Em AI engine in Java. This is a long term project, one in which I plan to invest at least two years. I'm still at college, haven't build anything ambitious yet and wanting to tackle a problem that will hold my interest in the long term. I'm new to the field of AI. From my data structures class at college, I know basic building blocks like BFS and DFS, backtracking, DP, trees, graphs, etc. I'm learning regex, studying for the SCJP and the SCJD and I'll shortly take a (dense) statistics course.

Questions:

-Where do I get started? What books should I pick? What kind of AI do poker playing programs run on? What open source project can I take a page from? Any good AI resources in Java? I'm interested in learning Lisp as well, is Jatha good?

Answer 1

Poker AI's are notoriously difficult to get right because humans bet unpredictably. It's usually broken into two parts.

1) Calculate the odds of your hand being the winner.

2) Formulate betting strategy based on 1.

I'd recommend starting with lots of statistics reading for part 1. It seems easy at first blush, but it's actually very complicated (and getting it wrong will doom your AI). Then move on to genetic algorithms for part 2. Betting strategies are mostly genetic algorithms. They adjust themselves based on past success and failures + some randomization so as not to become predictable.

Answer 2

The following may prove useful:

• The University of Alberta Computer Poker Research Group
• OpenHoldem
• Poker Hand Recognition, Comparison, Enumeration, and Evaluation
• The Theory of Poker
• The Mathematics of Poker

Answer 3

One interesting result I've heard is that if you restrict the betting options to fold, check and all-in, you can write an AI that wins one-on-ones with probability at least 49%, and 49.5% if it's (IIRC) not going first.

I don't know that this AI is easier to write that one which knows how much to bet, but it's food for thought: choosing amounts to bet only accounts for 1.5% of the probability of winning.

Answer 4

I wrote a Hold'Em AI in my undergrad. It wasn't particularly advanced, I used a Q-Value machine that traversed a number of states and updated Q values for each state.

I found the University of Alberta's AI Poker project an invaluable source of info for avoiding pitfalls.

As one poster above states, the first step is to nail in a couple of determinable poker rules - one-on-one poker can be developed programatically.

One pitfall I fell into was not building in reconfigurability early on. For instance being able to switch the grade of learning/playing.

I would be interested to hear how you get on drop me a mail stevekeogh at gmail.com

Answer 5

Also, letting genetic algorithm adjust the weights of neural network, which determines the decision logic. This approach is very suitable for poker AI.

I made my own AI like this. At first, I created ~1000 players, who didn't know how to play the game at all. Based on their initial luck during the hands, their fitness was weighted and new generation created. New "brains" were playing better than previous generation.

Eventually, the best individuals played very good.

Answer 6

As already recommended, the book Theory of Poker is a truly invaluable source of information for playing the game as well as for building an AI. You should probably buy it as it does not cost that much.

University of Alberta resarch group does the state-of-the-art at the moment, though they have stiff competition emerging every now and then. (Not all poker bots and AI research in the field is public because of the temptation to use one's results in internet poker, though that's forbidden.)

First you should decide what sort of poker are you going to tackle first. two player hold'em is pretty much solved, though the best humans still put up a real fight with the best AI's available. The AI has the main advantages over humans by having an unlimited flawless memory of past hands, flawless analysis of the patterns based on that and as they are machines, they don't tilt like almost all humans occasionally do.

Fixed Hold'em is probably the easiest to crack, so you might want to start with 1-1 fixed hold'em and then decide what you want to do next.

Here are some aspects which change the correct strategy (and your AI):

• A cash game is different from a tournament -The number of players makes the decisions different.
• Hold'em is not the only poker. Omaha, Stud and others exist and are widely played.
• Fixed Limit is different from Pot Limit, which is different from No Limit.
• To beat the best you need to cover a lot of very subtle things the best players think about when they play. To beat a low-stakes amateur game, none of these things count.

If you decide to go for No Limit Hold'em, you might want to check out three-book series Harrington on Hold'em and a book No Limit Poker - Theory and Practice. Having read quite a many books on poker, I can say these books combined with the Theory of Poker are quite enough.