views:

862

answers:

6

I've implemented an Elo rating system in a game. There is no limit for the number players. Players can join the game constantly so the number of players probably rises gradually.

How the Elo values are exactly calculated isn't important because of this fact: If team A beats team B then A's Elo win equals B's Elo loss.

Hence I've got a problem concerning the starting values for my rating system:

  • Should I use the starting value "0" for every player? The sum of all Elo values would be constant. But since the number of players is increasing there would be some kind of Elo deflation, wouldn't it?
  • Should I use any starting value greater than 0? In this case, the sum of all Elo values would constantly increase. So there could be an Elo inflation. The problem: Elo points lose value but the starting value keeps always the same.

What should I do? Can you help me? Thanks in advance!

+1  A: 
  1. Do players see this score?
  2. Will the players understand Elo?
  3. Will players continue to play if their score becomes negative?

I would start everyone out at some positive point value (10, 100, 1000, it doesn't matter). When two people of relatively capability play each other, the scores trade as expected. Where you need to concentrate is some sort of relative capability between two players.

Suppose, later on in the game's life, I have 25000 points, and you're a n00b with 100. I beat you, I gain nothing and you lose nothing. Why? Because I just pwned a n00b, that's why. There should be no advantage for a new player to take down a starting player. Also, even if you are in some point range, you should implement something where you can only earn so many points from a given player in a certain time range.

Obviously, this will be something that will be continually tweaked throughout your game's life time.

Jarrett Meyer
Thank you. Players see the score since it is the main criterion for the player ranking. What do you want to say with "Will the players understand Elo"? The Elo score shouldn't become negative. If it would in case of a loss, the loser doesn't get any subtraction but the winner gets his addition as normal.
The thing you mention concerning 25,000 points vs 100 points: That would be an inflation, right? So I just have to avoid inflation.
No matter what, when a new player comes into the game, you will have some amount of inflation - whether you start at 0 or 100. If I have 0, but can't go negative, if you win a contest, you will get points without taking away any. That is adding total points into the system.So once you acknowledge that you must have inflation, now you must decide on measures to curb that inflation. Adjusting awards/penalties based on the relative scores of players is the best way to adjust.
Jarrett Meyer
Actually, marco -- if you're using an ELO system, then if the 25000-point player beats the 100-point player, few points change hands. But if the 100-point player beats the 25000-point player, many points change hands. It's one big advantage of the ELO system.
Chip Uni
+1  A: 

I don't know if it is useful, but Mark Glickman's Ratings Page discusses some issues with Elo ratings, their declines, etc. (see the last few paragraphs there). Also see his rating system, the Glicko system, which seems to account for playing frequency and discusses rating reliability. Finally, his research page has a lot of papers discussing ratings and their reliability.

Hope that helps.

Alok
Thank you. I want to use the Elo algorithm. That's sure. But I don't know where to start: 0 or any positive number (e.g. 1000).
If you're worried about rating deflation/inflation, you will have to make some changes to the algorithm: such as a variable k factor, injection of rating points, or both. I haven't looked at the Glicko system in a lot of detail, but it deals with the shortcomings of the Elo algorithm. It's a tricky problem: good luck!
Alok
New idea: The starting value for the Elo score is always the current average. So if I have users with 0, 300, 200 and 150, the fifth user will get 163 to start. Would this be good?
I think you're better off with either injection of points, or with a variable K-factor, or with a combination of both. Statistically, I can't prove (yet) if your method takes care of inflation or deflation. By the way, Glicko system is Elo's algorithm, but also has a reliability factor, so in principle you could use it. It seems that it's already used in many places.
Alok
+6  A: 

You can start at zero and add a fudge factor to the displayed score to keep it above zero, or you can start at 1000 - they are the same thing. Yes, with the 1000 starting point you'll have an increasing number of total ELO points in the system but it will always be the same number per player on average - 1000. The starting value for Elo is always the current average. ELO is a zero sum game, points lost by player A are gained by player B.

When you set a starting point at 1000 what you are essentially saying is that the average player = 1000 pts. With a closed group of initial players (beta testers?) this is true, within that group average = 1000. But if the game is something you improve at with time then your closed group average player becomes highly skilled compared to someone who hasn't played.

Now when you assign a 1000 to a new player you are saying new average players = existing highly skilled average player. This is not true, they are likely to be much less skilled that your original closed group. So the new player loses points and your highly skilled players gain => inflation. What you would need to do is accurately assess the skill of new players and assign them a ranking that is more in keeping with their actual skill. This could be done be assigning them a "provisional ranking" for their first x games until you get a feel for their skill. When provisionally ranked only their ELO score would change, not those of the people they play. Once they join the real system the points they bring into the scored ELO would roughly equate to their actual skill and they wouldn't move up or down dramatically => no inflation or deflation.

In short: Provisional rankings

Andrew
Thank you very much! So I can give every player 1000 starting points. But new players have just a provisional score for the first X games. Correct so far? In these X games, they only gain or lose points as normal but their opponents don't. Right?
Yes, exactly. Then after a few provisional games thy should have a rating which roughly reflects their true ability. Upgrade them to normal player status, with this rating and from then on treat them the same as everyone else.
Andrew
Ok. But the starting value is still a difficult point. By choosing either very weak opponents or quite strong opponents - depending on whether the starting is higher or lower than the average - I can have a strong influence on my starting value after X games. Can't I?
And what to do with negative ratings? Allow them? Or just don't change the rating if it would become negative?
Well this is the same ELO system that is used in chess and they don't get negative ratings. If you start at the same starting point you shouldn't either. The further you move from the average the harder it becomes to keep moving away, there is an effective limit. In chess "average" = 1200 and "grandmaster" = 2500 so if you are able to move 1300 pts away from average you are a grandmaster. To get negative you'd need to move 1200+ pts away from the average - a "negative" grandmaster of poor play. It would be extremely difficult and I doubt it will ever happen.
Andrew
Perhaps the X games requirement should be "X games against opponents who are rated +/- 100 ELO points relative to your own ranking".
Andrew
In chess they don't get negative ELO scores because the FIDE drops users with scores less than 1200. ;) Why ever ... I could do this, too, right? Just drop (don't show) all scores less than 0.
Yes, you could. Personally I doubt anyone will ever manage to play badly enough to get their score down to zero. They have to lose every time they play and they have to find opponents with consistently lower ELO rankings in order to keep moving down. It would be quite a serious effort to go that low. If a player managed to go negative then perhaps a polite message suggesting that this isn't the game for them and perhaps they might like to try doing something else is needed!
Andrew
Ok, thank you. Last question :) What starting value should I take for the provisional scores? 0 or 1200?
I'd go with either 1200 or track what people usually end up at when transitioning from provisional to real score and give them the average of that. Basically try to measure how good the new players are.
Andrew
+1  A: 

This site used the elo rating system. They start at 1200

taken from http://gameknot.com/help%5Fratings.htm

GameKnot rating system is based on Elo rating system with a fixed K = 20 and the following modifications:

The first 20 games are used to establish player's rating on the website. During the first 20 games, the player's rating is calculated as an average of the ratings of all his/her opponents, +400 in case of a win, -400 in case of a loss, same for a draw. +/-200 points are used when playing against a player with a provisional rating. PLayer's rating is provisional during the first 20 games, after which it becomes established. Player's rating is considered to be equal to 1200 during the first 5 rated games.

Timeouts are counted as wins only if there were at least 3 moves made in the game (losses are always counted for the timed out players, regardless of how many moves were made).

The higher of two ratings, at the beginning of the game and at the end, is used to calculate the rating adjustments after the game is over.

For example, if during your first 20 rated games, you played 3 games and you won against 1200 player with provisional rating, then against 1400 player with established rating, but lost against 1600 player with established rating, your rating will be: ( (1200 + 200) + (1400 + 400) + (1600 - 400) ) / 3 = 1467

Or, if during your first 20 rated games, you win against 1200 provisional, win against 1400 established, lose against 1600 provisional, draw against 1500, your rating will be: ( (1200 + 200) + (1400 + 400) + (1600 - 200) + 1500 ) / 4 = 1525

Kieran
Thanks, very useful!
+1  A: 

I think that most ELO-like systems on the Internet will have a problem with ratings creep.

Assume that all new players start at a rating of zero. LousyPlayer loses a dozen games, and his rating goes far below zero. What stops him from clearing his browser, registering a new account under a new email address, and starting over?

If this is possible, then low-ranked accounts will lay fallow, moving up the practical average rating.

Chip Uni
Thank you very much, I didn't think of this. So I can't completely prevent inflation or deflation, right?
Nope! ELO works perfectly when everyone plays everyone else... but not when people move in and out of the system. You should look at what other gaming systems use to prevent inflation or deflation.
Chip Uni
+1  A: 

I understand the concepts of the ELO like systems. I also understand different ways on how an 'established' rating can be accomplished by comparing the results from provisional against established rated players. But what I'm interested in is how do I start out if none of the players have an established rating? I could assign them all a default starting value and take the first 20 results and start with that value as established. But my concern about this method is that these established ratings are all based on this fictional starting value. And even more important, players starting out after all players have an established rating could benefit from the fact that his rating will be based on opponents with established ratings. Are my concerns valid or should I neglect this fact?

Patrick Savelberg
I had these concerns, too. This could really be a problem. Especially, if you have ELO deflation or inflation in your system. Maybe you let people start with 1500 ELO points in the beginning. But then you have an enormous inflation so that 1500 points are now at the bottom of your ranking.