How much resources does consume sleeping thread?

Question

I'm wondering, how expensive it is to have many threads in waiting state in java 1.6 x64.

To be more specific, I'm writing application which runs across many computers and sends/receives data from one to another. I feel more comfortable to have separate thread for each connected machine and task, like 1) sending data, 2) receiving data, 3) reestablishing connection when it is dropped. So, given that there are N nodes in cluster, each machine is going to have 3 threads for each of N-1 neighbours. Typically there will be 12 machines, which comes to 33 communication threads.

Most of those threads will be sleeping most of the time, so for optimization purposes I could reduce number of threads and give more job to each of them. Like, for example. reestablishing connection is responsibility of receiving thread. Or sending to all connected machines is done by single thread.

So is there any significant perfomance impact on having many sleeping threads?

Answer 1

For most cases, the resources consumed by a sleeping thread will be its stack space. Using a 2-threads-per-connection-model, which I think is similar to what you're describing, is known to cause great scalability issues for this very reason when the number of connections grow large.

I've been in this situation myself, and when the number of connections grew above 500 connections (about a thousand threads), you tend to get into lots of cases where you get OutOfMemoryError, since the threads stack space usage exceeds the maximum amount of memory for a single process. At least in our case, which was in a Java on 32 bit Windows world. You can tune things and get a bit further, I guess, but in the end it's just not very scalable since you waste lots of memory.

If you need a large number of connections, Java NIO (new IO or whatever) is the way to go, making it possible to handle lots of connections in the same thread.

Having said that, you shouldn't encounter much of a problem with under a 100 threads on a reasonably modern server, even if it's probably still a waste of resources.

Answer 2

This won't scale very well. Having a large number of threads means the VM has to spend more time context-switching, and memory useage will be higher due to each thread requiring its own stack space. You would be better off with a smaller number of threads processing in a pipeline fashion, or use the thread pool with asychronous techniques.

Answer 3

Lots of threads equate to lots of stack space, which will eat your memory - check out your -Xss settings for just how much, then do the math.

And if you ever have to do a notifyAll() for some reason, then of course you are waking up loads of extra threads - though you may not need to do that in your proposed architecture.

I'm not sure you can easily avoid having one thread per listening socket in this model (though I know very little about NIO - which might fix even that issue), but take a look at the java.util.concurrent.Executor interface and its implementing classes for a decent way to avoid having too many additional threads hanging around. Indeed, the ThreadPoolExecutor might be a good way to manage your listening threads too, so you don't spend too much time creating and destroying threads unneccessarily.

Answer 4

We had very much the same problem before we switched to NIO, so I will second Liedmans recommendation to go with that framework. You should be able to find a tutorial, but if you want the details, I can recommend Java NIO by Ron Hitchens.

Swithcing to NIO increased the number of connections we could handle a lot, which was really critical for us.