Java Socket Programming does not work for 10,000 clients

Question

I can create multiple threads for supporting multi-client feature in socket programming; that's working fine. But if 10,000 clients want to be connected, my server cannot create so many threads.

How can I manage the threads so that I can listen to all these clients simultaneously?

Also, if in this case the server wants to send something to a particular client, then how is it possible?

Answer 1

This is not a simple question, but for a very in depth (sorry, not in java though) answer see this: http://www.kegel.com/c10k.html

---

EDIT

Even with nio, this is still a difficult problem. 10000 connections is a tremendous resource burden on the machine, even if you are using non-blocking sockets. This is why large web sites have server farms and load balancers.

Answer 2

Highly scalable socket programming in Java requires the selectable channels provided in the "New I/O", or NIO packages. By using non-blocking IO, a single thread can service many sockets, tending only to those sockets that are ready.

One of the more scalable open-source NIO applications is the Grizzly component of the Glassfish application server. Jean-Francois Arcand has written a number of informative, in-depth blog posts about his work on the project, and covers many subtle pitfalls in writing this kind of software with NIO.

If the concept of non-blocking IO is new to you, using existing software like Grizzly, or at least using it as a starting point for your adaptation, might be very helpful.

Answer 3

You should investigate Java's NIO ("New I/O") library for non-blocking network programming. NIO was designed to solve precisely the server scalability problem you are facing!

• Introductory article about NIO: Building Highly Scalable Servers with Java NIO
• Excerpts from O'Reilly's Java NIO book

Answer 4

A thread-per-connection threading model (Blocking Socket I/O) will not scale too well. Here's an introduction to Java NIO which will allow you to use non-blocking socket calls in java: http://today.java.net/cs/user/print/a/350

As the article states, there are plenty of frameworks available so you don't have to roll your own.

Answer 5

Why don't you process only a certain amount of requests at a time.

Let's say you want to process a maximum of 50 requests at a time (for not creating too many threads)

You create a threadpool of 50 threads.

You put all the requests in a Queue (accept connections, keep sockets open), and each thread, when it is done, gets the next request then process it.

This should scale more easily.

Also, if the need arise, it will be easier to do load balancing, since you could share your queues for multiple servers

Answer 6

The benefits of NIO are debatable. See Paul Tyma's blog entries here and here.

Answer 7

Thanks Martin, The problem is that I have used thread pool. Server can send response to client for once. If server wants to send response further and i have connected to another client then this will not work. Because as per client's view the server thread is closed it means server is disconnected.

So how to overcome this issue?

Answer 8

As previously mentioned, 10.000 clients is not easy. For java, NIO (possibly augmented with a separate threadpool to handle each request without blocking the NIO thread) is usual way to handle a large amount of clients.

As mentioned, depending on implementation, threads might actually scale, but it depends a lot on how much interaction there is between client connections. Massive threads are more likely to work if there is little synchronization between the threads.

That said, NIO is notoriously difficult to get 100% right the first time you implement it.

I'd recommend either trying out, or at least looking at the source for the Naga NIO lib at naga.googlecode.com. The codebase for the lib is small compared to most other NIO frameworks. You should be able to quickly implement a test to see if you can get 10.000 clients up and running.

(The Naga source also happens to be free to modify or copy without attributing the original author)

Answer 9

Personally I would rather use create a custom I/O non blocking setup, for example using one thread to accept clients and using one other thread to process them (checking if any input is available and writing data to the output if necessary).