views:

92

answers:

2

I am trying to proccess a queue of tasks from a database table as fast as possible while also limiting the number of threads to process the tasks.
I am using a fixed sized thread pool with Executors.newFixedThreadPool(N);

I want to know if there is a way of knowing if the thread pool is full, by that I mean are there currently 50 threads running, if so then I'll wait for a thread to be available before starting a new one instead of sleeping the main thread.

Code of what I would like to do:

ExecutorService executor = Executors.newFixedThreadPool(N);
ResultSet results;

while( true ) {
    results = getWaitingTasksStmt.executeQuery();

    while( results.next() && executor.notFull() ) {
        executor.submit( new thread( new runnableInheritedClass(results) ) );
    }
}
+3  A: 

The ExecutorService does all that work for you. If all of the threads are currently being used by other tasks, the new tasks will be placed into a queue and processed at some later time. Your main thread will not block when submitting a new task even if all the threads are currently being used.

ExecutorService executor = Executors.newFixedThreadPool(N);
ResultSet results;

while( true ) {
    results = getWaitingTasksStmt.executeQuery();

    while( results.next() ) {
        // If all threads are in use, the new task will be queued
        executor.submit( new runnableInheritedClass(results) );
    }
Matthew T. Staebler
+4  A: 

You should not submit a Thread object to the executor, that negates its entire purpose. You should submit Runnable objects and let the Executor worry about the Thread handling. It will automatically queue up your Runnables when all threads are busy and when one task is complete it will grab a waiting task from the queue.

So your code should look more like this:

ExecutorService executor = Executors.newFixedThreadPool(N);

ResultSet results = getWaitingTasksStmt.executeQuery();

while( results.next() ) {
    executor.submit(new RunnableInheritedClass(results) ) );
}

executor.shutdown();
executor.awaitTermination(10, TimeUnit.MINUTES);

This will allow 10 minutes for all tasks to complete, adjust as neccesary for your situatioin. Waiting forever is discouraged so think of some kind of reasonable timeout for your tasks.

Gerco Dries