views:

841

answers:

11

It recently emerged on a large poker site that some players were possibly able to see all opponents cards as they played through exploiting a security vulnerability that was discovered.

A naïve cheater would win at an incredibly fast rate, and these cheats are caught very quickly usually, and if not caught quickly they are easy to detect through a quick scan through their hand histories.

The more difficult problem occurs when the cheater exhibits intelligence, bluffing in spots they are bound to be called in, calling river bets with the worst hands, the basic premise is that they lose pots on purpose to disguise their ability to see other players cards, and they win at a reasonably realistic rate.

Given:

  • A data set of millions of verified and complete information hand histories
  • Theoretical unlimited computer power
  • Assume the game No Limit Hold'em, although suggestions on Omaha or limit poker may be beneficial

How could we reasonably accurately classify these cheaters? The original 2+2 thread appeals for ideas, and I thought that the SO community might have some useful suggestions.

It's an interesting problem also because it is current, and has real application in bettering the world if someone finds a creative solution, as there is a good chance genuine players will have funds refunded to them when identified cheaters are discovered.

+1  A: 

I doubt you can say with any certainty if someone is cheating or if they are just good at Poker, past a certain point.

You could however narrow the candidates who you think might be cheating, by looking at the users who over your time period benefited overall. This will remove the vast majority of users, allowing you to focus your resources better. (This of course will include users who are skilled at Poker.).

Once you've done that, you can compare the history of play from while the cheat was possible to the history afterwards or before, and see if the users success decreases or increases. That should give you a list of users who you need to investigate more carefully, possibly by analyzing specific games.

Enjoy, it's a nice problem.

Bravax
+2  A: 

I think if they cheat intelligent, so with winning not too much rounds, it won't be detectable. I don't believe you could see the difference between luck and cheating here.

But I would like to know at which online poker provider the cheating is possible. Because I can't imagine a way how to do this, if the poker software is coded properly. If I was asked to program an online poker software, The users wouldn't be able to see the opponents cards, because there is no way he could get this information. And this is how I would do this.

  1. Every connection between users and server is encrypted
  2. no communication between users, the users can only talk to the server.
  3. The server tells every user only the cards the user should see, and no other cards, unless the round is finished and the users open their cards.

The only way the users could cheat here is, you get together with other players, or impersonate multiple players with different accounts and accessing IPs, and open another channel to communicate between the players. This way the group has a big advantage because they know more than their own cards, but there's still no way they can see other cards. And because it's now a group that is cheating it is even more harder to detect it, because they can share their earnings with multiple players, and this group could even have a player that looses more than (s)he gains and still win overall.

jigfox
You would be suprised how incompetant some poker sites are. Recently UltimateBet was found to be transmitting users login details raw, with no encryption. This was happening for a long long time.
Tom Gullen
Wow.. Amazes me that someone would do that.
Chaoz
The same site also marked certain users (employees) with the ability to see opponents cards and collectively the stole millions, possibly tens of millions off other players.
Tom Gullen
A: 

To be perfectly honest, I'd doubt very much that the players who could see opponents hands were random. There must be some sort of cross over in the code that generates the card view that was selecting some users but not others. I would recommend running tests on this code and trying to find a trend in the "viewers" and "non-viewers". If you find a strong trend, then the trend could be applied to the actual dataset too see which users, or which hands or which whatever was generating the code fault.

Daniel Hanly
For this problem we should assume it isn't a purposefully coded code fault, although this is a possibility. We assume that there was a gap in their security which allowed hackers to gain entry and exploit the system without the operators of the site being aware.
Tom Gullen
in the UB scandal, this was a system-wide password which gave the user with that password superuser access. What a shame...
Alexandre C.
+4  A: 

Plot V$PIP versus winrate of all players with a statistically significant #hands played. You should see outliers with naked eye. I think that's the basic thing to do first.

Then you can plot WTSD vs winrate, winrate at showdown vs winrate without showdown, %won at showdown vs VPIP.

The stats you choose must be significant statistically. If you know poker, the above choices make sense.

This is not a job for a machine, outliers are detected by eye.

EDIT: Omaha is much tougher, since it is really variant. There are cases of unbelievable streaks made by weak players who were not cheating.

Alexandre C.
Excellent suggestion
Tom Gullen
Actually I think you can detect most cheats this way.
Alexandre C.
Wasn't infinite or near-infinite river aggression a tip-off to the UB situation?
Greg Bacon
you can definitely plot other statistics, river aggresion factor needs a lot of hands to converge though. But any cheater will tend to play more hands that she should do. Even 2/3% of extra VPIP is worth considering; if you actually look at plots of VPIP vs Winrate, there is a smooth density cloud your cheater will have a hard time trying to fit in.
Alexandre C.
You could apply the techniques for finding equilibrium strategies to finding cheater strategies (by allowing them know each other players hands) and then this would method for identifying cheaters would be far less useful if not render it outright worthless.
NickLarsen
@NickLarsen: Such a genius cheater, with credible statistics cited in the answer, should be able to play poker profitably anyway. Usually cheaters are poor poker players.
Alexandre C.
That's true, but it's important to find all cheaters, not just the most obvious ones, and to find them in a deterministic manner.
NickLarsen
@NickLarsen: Your only bet is a cop behind each player then. You will never get occasional players who collude via MSN or cell phone in any way. What you want to catch is regular cheaters, which will get caught one way or another with very simple statistical analysis.
Alexandre C.
That is not true. Related to the original post, cheating is defined as being able to see all of the other players cards which is preventable. It *is* important to identify **all** cheaters however, and there is no simple statistical analysis which will identify collusive behavior quickly or more importantly in a deterministic manner. Simple statistical models may help you narrow down the field of possible collusive cheaters, but there will be plenty of false positives and plenty of collusions which are not so easily identified.
NickLarsen
Don't get me wrong, I think using the easy method to identify the most likely incidents of collusion is a great idea from a business stand point, but that it is incomplete for a solution to the problem.
NickLarsen
I'm ready to place a rather big bet on the fact that such simple stat analysis will get *all* the players who can see *all* the opponent cards and play more than 50k hands while cheating (this is *not* much). Collusion detection is way tougher, since it involves only a few players at one table.
Alexandre C.
+2  A: 

Well. IT people get fascinated by all kinds of wrong question.

A better question is "how is cheating even possible ?". There is no need what so ever to send the opponent's hands over the wire until at showdown. If that data isn't sent to the client, then how could they cheat ?

They'd need to break into the server. Don't tell me that isn't preventable.

Erwin Smout
this doesn't rule out collusion, which is forbidden.
Alexandre C.
Another possibility is that the randomization/shuffling system is not at all random and the users have figured out this system and are using it to "know" what cards have been dealt...But yeah, either way, fixing the implementation would be the highest priority. Figuring out how to spot them in large data sets of historical game results seems like it boils down to just being a really interesting academic problem.
Eric Ryan Harrison
@Eric: A properly written poker server that have hundred of users playing at the same can be a near-true random number generator since the single pseudorandom number generator is used concurrently by multiple tables at random interval. It will be impossible to predict the next number by recording randomness data from a single table. If, on the other hand, you have a bot on every table recording everything, then probably, but that would assume that you have zero latency or the server are not suspicious of the many multiple connections from a single area.
Lie Ryan
@Eric: The only explanation left is that the server is sending the card's data before everyone opens the card for scoring. The game seems try to encrypt this card data, and someone found a way to break the encryption. This is totally wrong in the first place. A client should only receive information about its own cards and only receive other card data when they are actually dealt. This way, the only way to peek to other's card or to the undealt cards is to hack the server, which should raise the red flag.
Lie Ryan
+4  A: 

I hate to be so blunt, but all the answers on this page with the exception of @Erwin Smout's are worthless.

Statistical analysis is a joke for identifying poker cheats
I realize the question allows there to be millions of hands worth of history available to the system. I'm sure there are players with hand histories this large, hell, I've probably played this many online hands. But I've also been playing online for over 10 years. Thats not a small amount of time, and it is my understanding that two conflicting things are true when it comes to identifying online poker cheaters: it needs to happen in a small amount of time, and like any good thief, an online poker cheat is going to take his stash elsewhere immediately after the taking.

There was a great example of the variance in poker in this paper which was generated by matching an always raise player versus an always call player (page 13 of the PDF). Over the course of 100,000 hands, wayyyy more than I think most people would be willing to play against someone who could see their cards, the always call player won on average .026 small blinds per hand. I know this does not sound like much, but assuming stakes of $5-10, that comes out to $6,500. Maybe someone can help me find the link, but the measured professional win rate is less not too much larger than this. Please note, NEITHER of these players was cheating, and the statistically expected difference over this number of hands is significantly less than what actually transpired.

What online poker players need to understand
Poker is gambling. It is a game of skill, because some players are able to elicit more information from their opponents than their opponents are able to gather, and that extra information is often as useful as seeing other peoples cards. Even players who are better players than their typical opponents, will end up long term losers. If you do not understand this, you're just searching for witches with statistics in the arbitrarily small number of hands you'll be playing against any opponent.

What can be done?
Keeping in mind the question states that cheaters are able to see the other players cards, you don't need statistical analysis to identify them. There are only three ways in which that is possible.

First is that the server is sending the information intentionally to clients which is an obvious security issue and should not be implemented (IMO, even for moderators). If a site was found allowing this to happen, it is the player's responsibility to move their funds elsewhere, or refuse to play on the site until that terrible design decision is rectified. It should also be the responsibility of the sites to inform their players of the exact steps that take place during hands played on the site so they have that to make their decision on when choosing a site in the first place. Security by obscurity is unpermitable. As for catching the thieves, this information should be sitting in log files on their servers, which should be regularly audited for this type of behavior.

Second is that the user has hacked the poker server and they would know about that in hurry, or else once it is exposed, it is again players responsibility to determine where to play. In this case, the cheater can be prosecuted in most countries.

Lastly, it is possible the dealing algorithm has been cracked. This one was a major problem in the past with companies that used naive methods to deal hands, but most of the major shops solved this problem by taking random inputs from players logged into their system as well as using entropy generating hardware to seed their random number generator. Thats not to say it cannot be cracked however. If this is the case, the only option is for the company to engineer a new random number generator.

NickLarsen
In addition, redistributing funds back to players who lost to the cheaters is a terrible idea because it is often impossible to accurately determine the value stolen from each player. There have been some famous online poker cheats caught and the method for delivering funds to their opponents has often benefited players who avoided them (particularly in tournaments) than players who actually lost value to them.
NickLarsen
Great response. You seem to know a lot more about this topic than I do. Thanks for the post. Though I never understood the desire to cheat in online poker. I play a lot of online poker because I enjoy the game, but I would never even CARE if someone was actually cheating at a table I was sitting at. All of the online poker I play is free, so the chips are meaningless. I also feel that a lot of people play much more aggressively than they normally would just because the risk for bad play is much lower than it would in real life. Doesn't seem worth worrying about. Thoughts?
Eric Ryan Harrison
The stakes at the table define the game, more than the rules of how the cards are dealt. In online poker free money, your bankroll is effectively infinite, though they may only give you a small amount to begin with. Naturally people freely (no pun intended) play for higher stakes when they have more money. The game is the same though, the calculation is the exact same, and while the decisions you make in free money are different than the decisions in real money poker, they *should* still be built on the same principles.
NickLarsen
If you want to define cheaters as collusive players instead of being able to see the other players cards, then there are a number of techniques you can use to identify them however, and most sites have started implementing the most obvious defenses.
NickLarsen
@Nick, good answer but perhaps missing the point? We are not concerned with catching colluders. A security issue was discovered, theoretically giving hackers the ability to see opponents cards, and the poker site wants to determine if anyone did exploit this vulnerability. That is all.
Tom Gullen
A: 

I'm by no means a data-mining expert, and my grasp of statistical analysis of large data sets is pretty weak as well (and I'm not very good at poker, even though I love it) so take everything I say here with a grain of salt.

  1. Weed out the junk data. You are going to only really care about players that fit into two categories: (1) players who win more hands than they lose, (2) players who win more money than they lose. Who cares about a cheater who loses a lot? Heh.

  2. With this paired down list of players to actually analyze, I would take a look at their style of play. Assuming you have a lot of historical data, I would build a player skill profile and attempt to normalize their betting strategy. As a poor poker player, I normally will back up weaker cards that no decent player would back simply because they feel good. For example, any time I am dealt a face card with another low card (2, 3, 4, 5), if they're suited, I'll often ALWAYS call any bets made by other players before the turn, even though this strategy is not very successful. Pre-turn raises above the Big Blind often indicate a player has a pocket pair, yet my love of playing won't let me fold a suited hand pre-flop.

So for me, your analysis of my play would say that me matching aggressive calls pre-flop when I have anything suited would be normal. But a different player who only occasionally calls large pre-flop bets would be an indication that something might be out of whack.

I don't know what sort of system you'd need to build to make a profile of different users styles of play, but I imagine you could use some computer learning algorithms to "learn" a person's style of play with pretty decent accuracy.

  1. You mentioned that a smart user would throw hands to minimize his appearance as a cheater. I think this is a GREAT opportunity for more profiling. Would an experienced, winning player play through an awful hand? Probably not, ever. If I was dealt a 4S, 7H, and saw 9D, JC, AH on the flop, I would know that my chances of winning were really, really small. It also tells us that the cards given on the flop aren't very strong for anyone, so anyone at the table betting probably has a Jack or Ace paired, two pair, or three of a kind. Since you know your 4S, 7H is worthless, you'd either bet hard to bluff the pot or fold outright. Not very many good players (who would have been found in your winning players shortened list) would ever stick around on a hand like that.

Anyway, those are the things I've thought of. Now actually implementing them, I have no idea where to even begin so I'm afraid I can't be of much help there. This is a very interesting academic problem though, so please do us a favor and keep us informed of what you end up going with. If you want to take this conversation offline, feel free to email me at [email protected].

Eric Ryan Harrison
You only care about `players who win more money than they lose`, no other judgement is necessary because a losing cheater is not taking value from his opponents.
NickLarsen
Valid point @NickLarsen. I still think it might be important to at least be aware of players who win more hands than they lose if for no other reason than to possibly catch good bots that (currently) have bad betting algorithms. But you're right, the money aspect is certainly the most important thing to consider.
Eric Ryan Harrison
A: 

the most likely cheating situation would seem to be people working together. Three guys at same table knowing each others cards should be able to make some betting adjustments that would allow the pool of betters to come out ahead.

What stops are in place to prevent collusion?

This isn't what the question is about.
Tom Gullen
+1  A: 

For all of you expressing disbelief that this is even possible: the community on the poker forums linked in OP were similarly awestruck, but the site in question has confirmed that such a security vulnerability was present. Quite simply, the site was using very basic and insecure crypto to transmit hole card data to its players. Theoretically, it would have been possible for anyone aware of this to intercept transmissions from the site to a specific victim (eg. by being physically nearby and intercepting wireless data), and to cheat that player using the intercepted knowledge.

The question is about how to detect whether this vulnerability was actually exploited (before it was fixed), and if so by whom, given the resources outlined.

Oh, and also some of you seem to be assuming we're talking about a hypothetical scenario, and/or play-money poker; we're not. The site is real, the vulnerability was real, the investigation is really happening (see link in OP), and the games under investigation are real-money games with normal buyins of $200 and above.

karln
Thank's for explaining it a little better than me :)
Tom Gullen
A: 

The answer to your question is simple. There is no way to detect that type of cheater with just hand histories. You need the information that is not public in order to correlate multiple characteristic's to find a suspected cheater.

Ohh yea, and obviously the companies that provide these games do everything possible to setup shop in a low tax, non-regulated country. Until they are regulated and enforce strict code compliance and testing this will continue to happen.

c0mrade