Clustering using Threads in Java

Question

I have a job that takes too long time in Java. So I want to divide this job into threads and run them. After the threads finishes their jobs, returns to my service and Service give them new jobs. ThreadGroup is suitable for this or any other recommendation?

Answer 1

First of all, you need threads if either:

a) You have a multiprocessor machine

or b) You have a single processor but your jobs are IO-intensive (and not CPU-intensive)

Otherwise, you will gain nothing when using threads.

What you need here is ThreadPool

Answer 2

Take a look at the java.util.concurrent package.

There's a tutorial where you should find everything you need to know here:

http://java.sun.com/docs/books/tutorial/essential/concurrency/

Focus on the High Level Concurrency Objects in particular.

Answer 3

ThreadGroup isn't generally of much use to application code. It's not a great deal of use to container code either. The Java PlugIn uses ThreadGroup to distinguish which applet a thread belongs to.

java.util.concurrent, in particular ExecutorService, provides amongst other things handy utilities for handling threads and concurrency.

For computationally intensive fine-grained tasks, the fork-join framework in JDK7 will be useful.

Before starting on this difficult code, you might want to consider whether it is worth it. Can you do other optimisations that doesn't require large scale thread use? Is it I/O latency you are trying to deal with? If it is CPU-intensive, there is not a great deal of point in using many more threads than you have in hardware.

Answer 4

Not sure in what state of development your project currently is, since your problem statement is quite limited, but you might want to consider getting having a look at the fork-join project coming in JDK7: http://www.ibm.com/developerworks/java/library/j-jtp11137.html

There's a lot to gain & learn from looking at that, and since it's all open source you can already download the code as a patch and have a go at working with it.

(Might not be applicable for anything you have to implement right now, but worth a look non the less if you intend to develop / maintain your application for some time in the future)

Answer 5

Check out the ExecutorCompletionService - it does exactly this.

Example: [pulled from Java 6 API JavaDocs]

 void solve(Executor e, Collection<Callable<Result>> solvers)
     throws InterruptedException, ExecutionException {
       CompletionService<Result> ecs
           = new ExecutorCompletionService<Result>(e);
       for (Callable<Result> s : solvers)
           ecs.submit(s);
       int n = solvers.size();
       for (int i = 0; i < n; ++i) {
           Result r = ecs.take().get();
           if (r != null)
               use(r);
       }
   }