Which Java blocking queue is most efficient for single-producer single-consumer scenarios

Question

I'm working on a standard Java system with critical timing requirements for my producers (1/100s of ms matters).

I have a producer placing stuff in a blocking queue, and a single consumer later picking up that stuff and dumping it to a file. The consumer blocks when data is not available.

Obviously, blocking queue is the appropriate interface, but which actual implementation should I choose if I want to minimize the cost to the producer? I wan to play as little as possible on things like locking and allocating when I am putting stuff in the queue, and I don't mind if the consumer has to wait a lot longer or work a lot harder.

Is there an implementation that can be faster because I only have a single consumer and single producer ?

Answer 1

If the timing requirements are that tight, you'll probably need to do extensive benchmarking on exactly the same hardware to determine the best fit.

If I were to guess, I'd go with ArrayBlockingQueue because array-based collections tend to have nice locality of reference.

Answer 2

LinkedBlockingQueue will have O(1) insertion cost unless there is a memory allocation delay. A very large ArrayBlockingQueue will have O(1) insertion cost unless the system is blocked due to a garbage collection; it will, however, block on insert when at capacity.

Even with concurrent garbage collection I'm not sure if you should be writing a real-time system in a managed language.

Answer 3

ArrayBlockingQueue will probably be faster as you do not need to create link nodes on insertion. Note, however, that ArrayBlockingQueue is of a fixed length, if you want your queue to grow arbitrarily large (at least to Integer.MAX_INT) you will have to use a LinkedBlockingQueue (unless you want to allocate an array of Integer.MAX_INT elements).

Answer 4

Well, there really aren't too many options. Let me go through the listed subclasses:

DelayQueue, LinkedBlockingDeque, PriorityBlockingQueue, and SynchronousQueue are all made for special cases requiring extra functionality; they don't make sense in this scenario.

That leaves only ArrayBlockingQueue and LinkedBlockingQueue. If you know how to tell whether you need an ArrayList or a LinkedList, you can probably answer this one yourself.

Note that in LinkedBlockingQueue, "linked nodes are dynamically created upon each insertion"; this might tend to push you toward ArrayBlockingQueue.

Answer 5

I agree with Andrew Duffy's remark that implementing such a time-constrained system in java may be a mistake. Regardless, assuming you're married to the JVM, and you don't expect this queue to be running full (ie, the consumer can handle the load), I think you might be best served by a custom implementation, similar to an ArrayBlockingQueue but trimmed down/optimized for the single producer/consumer scenario. In particular, I'm liking the notion of spinning on the producer-side to wait for space, rather than blocking.

I referred to the java.concurrent sources for guidance, and drafted up this algorithm...

It looks pretty good to me, but it's entirely untested and may not be any faster in practice (probably not revolutionary, but I did crank it out myself :-). I had fun with it, anyway... Can you find a flaw?

Pseudocode:

private final ReentrantLock takeLock = new ReentrantLock();
private final Condition notEmpty = takeLock.newCondition();
private final E[] buffer;
private int head = 0
private int tail = 0;

public void put(E e) {
  if (e == null) throw NullPointerException();

  while (buffer[tail] != null) {
    // spin, wait for space...  hurry up, consumer!
    // open Q: would a tight/empty loop be superior here?
    Thread.sleep(1);
  }

  buffer[tail] = e;
  tail = (tail + 1) % buffer.length;

  if (count.getAndIncrement() == 0)  {
    sync(takeLock) { // this is pseudocode -- should be lock/try/finally/unlock
      notEmpty.signal();
    }
  }
}

public E take() {
  sync(takeLock) {
    while (count.get() == 0) notEmpty.await();
  }
  E item = buffer[head];
  buffer[head] = null;
  count.decrement();
  head = (head + 1) % buffer.length;

  return item;
}

Answer 6

You can write a latency sensitive system in Java but you have to be careful of memory allocations, information passing between threads and locking. You should write some simple tests to see how much time these things cost on your server. (They vary bases on the OS and the memory architecture)

If you do this you can get stable timings for operations up to 99.99% of the time. This is not proper real-time, but it may be close enough. On a trading system, using Java instead of C costs might cost £100/d, however the cost of developing in C/C++ rather than Java is likely to be much higher than this. e.g. in terms of the flexibility it gives us and the number of bugs it saves.

You can get fairly close the same amount of jitter you would see in a C program doing the same thing. Some call this "C-like" Java.

Unfortunately it takes about 8 micro-second to pass an object between threads in Java via a ArrayBlockingQueue on the servers I work on and I suggest you test this on your servers. A simple test is to pass the System.nanoTime() between threads and see how long it takes.

ArrayBlockingQueue has a "feature" where it creates an object on each element added (though there is not much of a good reason to do so) So if you find a standard implementation which doesn't do this, let me know.