Java threadpool functionality

Question

Hi stackoverflow,

I need to make a program with a limited amount of threads (currently using newFixedThreadPool) but I have the problem that all threads get created from start, filling up memory at alarming rate.

I wish to prevent this. Threads should only be created shortly before they are executed.

e.g.: I call the program and instruct it to use 2 threads in the pool. The program should create & launch the first 2 Threads immediately (obviously), create the next 2 to wait for the previous 2, and at that point wait until one or both of the first 2 ended executing.

I thought about extending executor or FixedThreadPool or such. However I have no clue on how to start there and doubt it is the best solution. Easiest would have my main Thread sleeping on intervals, which is not really good either...

Thanks in advance!

Answer 1

Have you tried taking a look at ThreadPoolExecutor ? Using the right constructor parameters, you could easily tweak the number and keep-alive time of the created threads.

Answer 2

I don't think you should manipulate the thread pool implementation. If you create the threads shortly before execution, you lose the main benefit of the pool, that recycles your threads.

Maybe you should reduce the maximum number of threads in the pool. If you instruct the pool to create too many of them, the total out-of-heap memory used for their stack spaces will consume all available memory. I assume that this is the kind of OutOfMemoryError you have (?).

Answer 3

If you're looking at this from a performance perspective, then it's best to take the hit in memory when you first start up the application than constantly get bombarded with allocating and deallocating memory while the program is running.

If it's using too much memory when you start the application, then it will still be too much memory later. You should throttle down the size of the thread pool.

There are additional benefits to using a thread pool, such as if you lose a thread along the way, the thread pool will automatically create a new one to replace it, keeping your thread pool at a constant size.

If this isn't the type of benefit that you're looking for, then you may wish to handle the threads in memory manually, and avoid the thread pool.

Answer 4

Looking at the details in your post...

I call the program and instruct it to use 2 threads in the pool. The program should create & launch the first 2 Threads immediately (obviously), create the next 2 to wait for the previous 2, and at that point wait until one or both of the first 2 ended executing.

Your problem is much more about synchronizing tasks execution than in fact pooling threads. From what you say here, you want to have 2 threads executing any number of tasks; if you don't want to have 100 jobs running at the same time, don't create a 100 threads pool...

I would suggest either using a BlockingQueue to control your Runnables, or create a 2 threads pool using a ThreadPoolExecutor, and feed it all your tasks. It will execute them when threads are available.

Does that make sense with what you try to achieve here?