Which protocol to choose for a turn-based game server

Question

I am writing a game server for a turn-based game in Java. These are the facts:

• The speed of the game is slow, so clients need to send data let's say every 8 seconds, and that data is most of the time only a small incremental update (a few dozen bytes), aside from situations like join of the game or list available games etc.
• The server must support a large number of players which, let's say 1000, which play one of a few hundred games
• When player makes a turn, other players in the same game must be notified of the move. The maximum number of players in the game is around 10

First of all, I excluded UDP from my option list as I need a reliable protocol because in rare situations I really need to send some data which cannot fit in one packet and I don't want to bother with merging packets and similar things, tracking the order of the arrived packages and other low-level stuff.

So the dilemma is whether to use TCP or HTTP.

TCP attempt #1

The connection from the client to the server (and vice-versa) is opened all the time. This way, when a player makes a move, the server can easily notify other players in the game which move was made. The main thing which bothers me with this approach is whether it is advisable or even possible to have as much as 1000 connections and sockets opened all the time?

TCP attempt #2

The alternative which I thought of is, to use to establish a seperate connection/socket on every request from a client. A client would open a connection, send some small data to the server and close that connection. With this approach, I can have a fixed size thread pool of let's say 10 and process client's requests in each thread separately so that there is at most 10 connectinos/sockets opened at any time. But there are two things which bothers me with this approach:

1. expensiveness of opening/closing the connection to the client
2. the way of notifiend other players in the game, since the connection to them is most probably closed. Each of them should in that case "poll" server for the update let's say every one second.

What is the cost of establishing a TCP socket/connection? Is this an expensive operation or this is done in only a few ms (or less)?

HTTP

1. Would there be a lot of overhead if I would be sending a new GET/POST just to send a few bytes?
2. Can I keep 1000 HTTP connections to clients simultaneously and then use AJAX or similar things to reduce overhead? In that case, would 1000 simultaneous connections pose a significant problem regarding bandwidth/performance?

I am opened to suggestions/advice of any kind.

Answer 1

Establishing a TCP socket is a fairly cheap operation. In fact, the general HTTP model is to do just this. You should not ever keep HTTP sockets open continuously. Ajax calls, HTTP calls, these are designed to be opened and closed as quickly as possible so that the next request can be handled.

I don't see why the polling design wouldn't be ideal here. Poll when the user leaves an instance of a game back to the main game list for the current status. Poll every 15 second or so when the user is at the main game list. Make sure that the server handles this poll quickly, fast - less than a millisecond if possible.

Many web servers have a hard limit of 256 connections at once, although recently I've seen this up at 1024 on some servers. Regardless, you should never be coming close to this limit. If your software is optimized well for speed, no connection should be open for more than a millisecond or two, and there should be no way to even get close to the number of users required to use 256 sockets.

The only speed issue here is how long your server software takes to actually do the polled queries. The overhead of establishing the socket and closing it is nothing next to the overhead of the server-side code that you write.

Answer 2

A server is supposed to be able to have around 20'000 sockets open at the same time. If you decide to use http, you can use the new comet features of tomcat 6+ or jetty 6+, otherwise a thread will be allocated to each request.

Answer 3

I see that you look at the very "low level". Have you tried to use something at the higher level, something like http://code.google.com/p/kryonet/ (also developed for games) ? (and found maybe bad performance? )

I think the results delivered by KryoNet are quite good and it's very fast to program with their API.

Answer 4

HTTP IMHO. You will be able to pass through any proxy. The authentication could be done simple and once using HTTP sessions or cookies.

Do not worry about the server capabilities - most modern servers can handle thousands concurrent clients.

Answer 5

Just for your information: HTTP is TCP. A specific protocol that uses TCP, that is. HTTP is based upon TCP, just like TCP is based on IP, etc. So really your choice is between HTTP over TCP or a custom protocol over TCP. You're right that UDP is a poor match here.

If you're writing the server yourself, many of the benefits of using HTTP go away. HTTP's main benefit is that there are highly optimised servers already available so you can use it as a simple and effective RPC system. But if you're writing the server yourself you're not likely to reach the efficiency of the likes of Apache so you have to ask why you wouldn't just pick a simpler protocol to use? Besides, hacking around the pull-only nature of HTTP would seem to be the wrong way to go.

With this in mind, I'd just use a more lightweight protocol over TCP. You get more control over the connections and can notify interested clients of updates without requiring them to poll for changes. You also are able to lose the HTTP request/response overhead which is mostly superfluous in your case. You can instead use a fairly simple bespoke protocol, perhaps based around XML or JSON, or maybe one of the existing RPC methods available.

Answer 6

Just use TCP sockets, one persistent connection for each client, along with a thread to do I/O. Thread overhead isn't too high (Linux allocates by default 48k for new stacks, so this would take 48 megs for 1k clients, Windows allocates 2k IIRC), your code will be much more clean and easy to follow, and you will auto-scale with CPU cores. If you're concerned about firewalls, investigate HTTP CONNECT and HTML 5 WebSockets.

Answer 7

I would suggest that if you are doing a turn based multiplayer game with fairly small (<50K) game packets, you should consider looking at using XMPP/Jabber. Here are some reasons I think why

• Higher level protocol (XML) like HTTP where you don't have to deal with bits and bytes if you don't want to.

• Built it presence, lobby (with MUC), pubsub mechanism, user management/authentication, chat, etc. The list goes on...

• Don't have to worry about writing a scalable server the yourself. Most Jabber servers supports plugins. Just write the plugin and let the server scale it for you; a bit like a HTTP server

• XMPP is an extensible protocol. You can carry your game data as part of a chat playload. Firewall friendly and most servers support BOSH

• Close to realtime and fairly reliable.

• Free and open source client (smack) and server (openfire) - in Java

Answer 8

Your "Attempt #1" is fine - there is nothing wrong with having 1000 open connections (it's not uncommon for a single IRC server to have more than 100,000 simultaneous open TCP connections).

(You may find that some OS settings need to be tweaked as you get near that number - for example UNIXes typically have a default, per-process open file limit around that, but it is simple enough to change).