I have a server application developed using C++ running on Windows platform. This server uses Windows sockets to communicate with clients using TCP. The socket server architecture inside the application is quite simple. The application pools 'x' number of threads during startup. Whenever a new client connects to the server, one of the thread from the pool is used for servicing this client. This design works fine even when 750-1000 clients connect to the server with just 20 threads to service the clients in the pool. The reason I beleive this works even when there are more than 20 concurrent client connections matching the 20 threads is because Windows queues all incoming client connections for a short period of time before sending out the connection refused or connectio timed out error. During this queued time, one of the service threads from the pool will become available to process the queued client connection. Now, my questions are, Is this design scalable? What is the max limit for the internal Windows socket queue? Can I accomodate tens of thousands of clients by increasing thread pool size by 100s. Does Windows even permit applications to have 100+ threads? Alternatively, how are applications like IIS web server designed? How many concurrent client connections does a single server like IIS handle?
A:
Take a look at I/O Completion Ports; they're how IIS is designed. They allow you to scale a server to 10,000s of concurrent connections with very few threads on relatively modest hardware.
I've written some articles on this kind of architecture and I have a free framework that allows you to cut to the chase and write your business logic without needing to worry too much about the detail of the I/O completion port stuff to start with; the framework includes all source so you can dig in and customise it once you have something working.
The free server framework can be found here and this includes source code and links to the original articles that I wrote about it.
Len Holgate
2008-12-21 18:39:43