Maintaining many socket connections with a single thread

Question

Many tutorials on socket communication I see seem to use 1 thread per socket. But on a server used for online gaming, you might have 10k concurrent users - 10k threads isn't probably a wonderful idea. I came across a tool (SmartFox) which claims to use a single thread for monitoring all socket connections, potentially thousands of them. This app happens to be in Java, but I figure C++ or C# could do the same... how would you achieve this?

Answer 1

Implement a queueing system with one thread polling the network and x threads acting as workers. You will need to implement a critical section around the code which dequeues and queues the connections.

Answer 2

If you are using C++, have a look at boost::asio.

Making your own is fun too, of course.

Answer 3

The C10K problem talks about this question.

Answer 4

Since you mentioned C++...

If you're on a Windows platform then you should be looking at I/O Completion Ports for this kind of scalability. I/O Completion Ports allow you to perform asynchronous I/O on sockets (and other devices) using a small number of threads to service many thousands of I/O operations (i.e. connections).

The way this works is that the I/O Completion Port is, essentially, a queue but the operating system optimises how threads are released to work on work items within that queue to prevent too many threads being released at once and to ensure that a thread that has just been used is more likely to be used again. See here: http://msdn.microsoft.com/en-us/library/aa365198(VS.85).aspx for the MSDN information on IOCP and here: http://www.lenholgate.com/archives/000637.html for the source code to my free client/server framework that uses IOCP under the covers.

As an example of the scalability possible, in this blog posting (http://www.lenholgate.com/archives/000568.html) I detail how I was able to achieve over 70,000 concurrent connections on a Windows Server 2003 machine with only 760MB ram.

Note that the C# async socket operations use IOCPs under the hood.