views:

1542

answers:

6

I need to execute some amount of tasks 4 at a time, something like this:

ExecutorService taskExecutor = Executors.newFixedThreadPool(4);
while(...) {
    taskExecutor.execute(new MyTask());
}
//...wait for completion somehow

How can I get notified once all of them are complete? For now I can't think about anything better than setting some global task counter and decrease it at the end of every task, then monitor in infinite loop this counter to become 0; or get a list of Futures and in infinite loop monitor isDone for all of them. What are better solutions not involving infinite loops?

Thanks.

+4  A: 

Basically on an ExecutorService you call shutdown() and then awaitTermination():

ExecutorService taskExecutor = Executors.newFixedThreadPool(4);
while(...) {
  taskExecutor.execute(new MyTask());
}
taskExecutor.shutdown();
try {
  taskExecutor.awaitTermination(Long.MAX_VALUE, TimeUnit.NANOSECONDS);
} catch (InterruptedException e) {
  ...
}
cletus
Looks nice and simple, only I have feeling that shutdown here is used not for what it was designed (or it is fine?).
serg
It's a common pattern. Transient threadpools like the above are good for several reasons and I pretty much always prefer them to persistent threadpools where a lot more can go wrong or at least it can be harder to figure out. The above is really straightforward which is why I like it.
cletus
this is exactly what shutdown / awaitTermination are meant for
matt b
It is a good pattern if this task handling is a one-time event. If this is done repeatedly during the same runtime, however, it is not optimal, as you would create and tear down threads repeatedly every time it is executed.
sjlee
+5  A: 

Use a CountDownLatch:

CountDownLatch latch = new CountDownLatch(totalNumberOfTasks);
ExecutorService taskExecutor = Executors.newFixedThreadPool(4);
while(...) {
  taskExecutor.execute(new MyTask());
}

try {
  latch.await();
} catch (InterruptedException E) {
   // handle
}

and within your task (enclose in try / finally)

latch.countDown();
ChssPly76
There aren't 4 tasks. There are "some number of tasks" done 4 at a time.
cletus
Only instead of CountDownLatch(4) I need to use number of total threads created, right?
serg
Sorry, I misunderstood the question. Yes, number of tasks should be the argument to CountDownLatch constructor
ChssPly76
I find this solution more elegant than the others, it looks like it was made for this purpose, and it's simple and straightforward.
wvdschel
What if you don't know the number of tasks before you start?
cletus
@cletus - then you don't use a CountDownLatch :-) Mind you, I'm not arguing that this approach is better than yours. However, I found that in real life scenarios I _do_ know the number of tasks, thread pool settings _do_ need to be configurable per deployment, and pools _can_ be reused. So I usually have thread pools injected by Spring and setting them as prototypes and manually shutting them down **only** in order to wait for threads to finish seems less than ideal.
ChssPly76
+1  A: 

You could wrap your tasks in another runnable, that will send notifications:

taskExecutor.execute(new Runnable() {
  public void run() {
    taskStartedNotification();
    new MyTask().run();
    taskFinishedNotification();
  }
});
Zed
A: 

You could use your own subclass of ExecutorCompletionService to wrap taskExecutor, and your own implementation of BlockingQueue to get informed when each task completes and perform whatever callback or other action you desire when the number of completed tasks reaches your desired goal.

Alex Martelli
+3  A: 

ExecutorService.invokeAll() does it for you.

ExecutorService taskExecutor = Executors.newFixedThreadPool(4);
List<Callable<?>> tasks; // your tasks
// invokeAll() returns when all tasks are complete
List<Future<?>> futures = taskExecutor.invokeAll(tasks);
sjlee
A: 

The CyclicBarrier class in Java 5 and later is designed for this sort of thing.

Pekka Enberg