.NET Client/Server Scaleability & Asynchronous I/O - Excess Thread Issues

Question

I have a pretty simple question which perhaps someone familiar with Server/Client design & the Asynchronous I/O paradigm of .NET could answer quickly...

I'm writing a server-side application which is designed to run on relatively non-sophisticated hardware (read: not-so-modern, average office desktop PCs), but accommodate a reasonably large number of users (though some users may be idle) - Because of this, I am concerned about the scalability impact of having a single thread for each client (due to excess context switching grinding the machine to a halt). This of course requires careful thought as to when a thread truly should be spawned.

One solution I've considered is to have a single master thread (separate from the connection listener thread) in the Server App which tracks activity on open connections by regularly going down the list of open sockets and performing a select(...) on each. If the thread finds incoming data, it sets up an asynchronous read (to consume the message, process it and return a reply, if necessary), and then moves onto the next socket. This process repeats for however long the server has at least on client connected.

In doing so, the maximal # of threads that are actually running is limited to only the number of clients actually communicating (and never higher than the maximal number of users connected); For clients that are idle, there is no idle thread which is simply taking up the CPU and bogging down the machine - The only time a thread is associated with a client is when there is actual data to be received & processed.

In my mind, this makes sense, though I guess this is the first time I've ever attempted something of this manner in C#. Does anyone have any thoughts on any issues this might bring about, or suggestions on maybe a better way of doing the same task?

Thanks!

Answer 1

One approach you may want to consider is using the ThreadPool to have a pool available threads for your applications.

This approach would differ from what you are considering in that there would be a fixed amount of threads your server could ever handle at the same time, and any requests received after that number is exceeded would be queued until another worker thread becomes available.

This is, in fact, what ASP.NET is using for incoming HTTP requests.

Answer 2

Have you considered using WCF? If you are concerned about context switching swamping the cpu, it might be better to use a call-and-close approach. WCF can greatly simplify your client-server communications, as you won't even have to worry about socket programming at all. It provides a wide variety of communication options out of the box as well, including HTTP, TCP, Named Pipes (same machine only), and MSMQ.

Depending on exactly how your conversations between client and server work, you may be able to go with MSMQ and pub/sub messaging, which would allow your clients to send messages asynchronously, and get asynchronous replies. This would scale to almost any load. If you have stateful conversations, WCF supports this as well, and if the server needs to call back to the client, duplex channels are available for dual-way, multi-message conversations.

WCF will handle threading, queuing, and resource management for you, allowing you to focus on the critical business behaviors. As for throughput, I recently did a simple, raw performance test of WCF using a super basic calculator service (add, sub, mul, div methods, basically zero behavioral overhead). On my developer workstation at work, a Core 2 Duo with a couple gigs of ram, we achieved 30,000 calls per second with four other developer machines hitting it. In contrast, a more modern machine, my home system with a Core i7 920 (8 logical cores), 12 gigs of ram, and gigabit ethernet, I wasn't even able to saturate the pipe or CPU's with 4 other systems hitting to it, and achieved nearly 100,000 calls per second. I can't even imagine the raw throughput if I could manage to saturate the pipe and/or cpu.

I think thats a strong indication that the overhead of WCF itself is ridiculously low, and shouldn't pose a problem for you, even on older hardware.

I highly recommend looking into it. It could save you a considerable chunk of time developing and maintaining a custom communications infrastructure.