Pre-creating objects in a separate thread

Question

Suppose I have a Java method like:

public Ouput respond(Input input) { /* .. */ }

The Output object has many fields - some of those fields depend on the Input object but the rest are pre-determined. I want to make the thread which calls respond() to return as fast as possible.

For this I want to spawn another thread which pre-creates Output object, sets some of the fields and puts them in a queue so that the thread running respond() can pick it up from the queue, set the remaining fields and return it.

What is the best way to implement something like this? My current prototype uses a bounded LinkedBlockingQueue but are there better ways to implement this idea?

My aim is to get the respond() method as fast as possible, so other suggestions which meet that goal are also welcome. :-)

Answer 1

Maybe you can adapt your implementation that Output-objects do not have to be created for every respond()-call but to recycle Output-objects when they are not needed anymore. Resetting fields should be faster than creating them from scratch and if you have a high object-throughput it also saves you some time in the garbage-collection and you can also improve cache-performance of the processor and Java VM.

Of course this can only work if you can control the whole process and can change the caller who gets the result of respond().

But if you really have such high performance-requirements this could be another idea to play around with. Just don't make it too complicated to be maintainable anymore.

Answer 2

This sounds like a case where you could make good use of Futures and Executors. See more here:

• http://java.sun.com/javase/6/docs/api/java/util/concurrent/Future.html
• http://java.sun.com/javase/6/docs/api/java/util/concurrent/Executor.html

Here's a concrete example:

ExecturService executor = Executors.newSingleThreadExecutor();

final Input input = ...
Future<Output> future = executor.submit(new Callable<Output>() {
  public Output call() {
    return respond(input);
  }
});

// do some work

Output output = future.get();  // this blocks until it's done

In your case, since you mentioned that some of the fields in the Output object are to be used immediately, while others are used at a later point, you could structure your Output object to contain futures as fields. For example:

public class Output {
   private String fieldA; // immediate field
   private Future<String> fieldB; // delayed field

   public void setFieldA(String fieldA) { this.fieldA = fieldA; }
   public String getFieldA() { return fieldA; } 
   public void setFieldB(Future<String> fieldB) { this.fieldB = fieldB; }
   public Future<String> getFieldB() { return fieldB; }
}

You'd then structure your respond method like this:

public Output respond(final Input input) {
   Output output = new Output();
   String fieldA = ...
   output.setFieldA(fieldA);

   Future<String> fieldBFuture = executor.submit(new Callable<String>() {
     public String call() {
       String fieldB = ...
       return fieldB;
     }
   }

   output.setFieldB(fieldBFuture);
}

And to use your Output object:

Input input = ...
Output output = respond(input);

String fieldA = output.getFieldA();
// do some work
String fieldB = output.getFieldB().get();

Answer 3

If the rest of the fields are truly "pre-determined" why not make them static on the Output class?

Answer 4

Why not create an object-creation factory that, on construction, pre-populates an internal store of Output objects, doles them out on request, and contains logic to know when it's necessary to repopulate the internal store? Perhaps something like this (note that this is just off-the-top-of-my-head simplecode; there are clear places where you can do a lot more, like have specific algorithms to determine when the pool has too few objects, synchronize on repopulation, etc.):

public class OutputFactory {

    private List<Output> outputPool = new LinkedList<Output>();

    public OutputFactory() {
        this.populatePool();
    }

    private void populatePool() {
        // spawn a new thread here, if you wish, to optimize time-to-return
        for (int i = 0; i < 100; i++) {
            outputPool.add(new Output());
        }
    }

    public Output getNewOutput() {
        Output newOutput = outputPool.remove(0);
        if (outputPool.size() < 10) {
            populatePool();
        }
        return newOutput;
    }

}

Answer 5

Since all threads are sharing the same heap, I can't imagine that creating them on a different thread is going to make much of a difference as far as allocation is concerned. If you are seeing a benefit from threading, it would be due to whatever computation is required to set the pre-determined fields on the Output object.

If that's the case, you might do even better to have a single template Output object with the predetermined fields pre-filled, and then simply have respond() clone the shared template object. (If the object is not already Cloneable and you can't make it so, it would cost about the same to copy the fields yourself.)

If the template thing doesn't work, but you could recycle Output instances, that's probably an even better option, as no new memory needs to be allocated at all, and the predetermined fields don't even need to be copied; they will already be set.

Either of those strategies would allow you to speed up the respond() method without the overhead of a separate thread.

Answer 6

Unless creating the object used by fields of Output are fairly expensive, the cost of using another thread will be many times greater than the cost of setting a few fields.