Java NIO Threading issue with SocketChannel.write()

Question

Sometimes, while sending a large amount of data via SocketChannel.write(), the underlying TCP buffer gets filled up, and I have to continually re-try the write() until the data is all sent.

So, I might have something like this:

public void send(ByteBuffer bb, SocketChannel sc){
   sc.write(bb);
   while (bb.remaining()>0){
      Thread.sleep(10);
      sc.write(bb);          
   }
}

The problem is that the occasional issue with a large ByteBuffer and an overflowing underlying TCP buffer means that this call to send() will block for an unexpected amount of time. In my project, there are hundreds of clients connected simultaneously, and one delay caused by one socket connection can bring the whole system to a crawl until this one delay with one SocketChannel is resolved. When a delay occurs, it can cause a chain reaction of slowing down in other areas of the project, and having low latency is important.

I need a solution that will take care of this TCP buffer overflow issue transparently and without causing everything to block when multiple calls to SocketChannel.write() are needed. I have considered putting send() into a separate class extending Thread so it runs as its own thread and does not block the calling code. However, I am concerned about the overhead necessary in creating a thread for EACH socket connection I am maintaining, especially when 99% of the time, SocketChannel.write() succeeds on the first try, meaning there's no need for the thread to be there. (In other words, putting send() in a separate thread is really only needed if the while() loop is used -- only in cases where there is a buffer issue, perhaps 1% of the time) If there is a buffer issue only 1% of the time, I don't need the overhead of a thread for the other 99% of calls to send().

I hope that makes sense... I could really use some suggestions. Thanks!

Answer 1

Prior to Java NIO, you had to use one Thread per socket to get good performance. This is a problem for all socket based applications, not just Java. Support for non-blocking IO was added to all operating systems to overcome this. The Java NIO implementation is based on Selectors.

See The definitive Java NIO book and this On Java article to get started. Note however, that this is a complex topic and it still brings some multithreading issues into your code. Google "non blocking NIO" for more information.

Answer 2

You don't need the sleep() as the write will either return immediately or block. You could have an executor which you pass the write to if it doesn't write the first time. Another option is to have a small pool of thread to perform the writes.

However, the best option for you may be to use a Selector (as has been suggested) so you know when a socket is ready to perform another write.

Answer 3

For hundreds of connections, you probably don't need to bother with NIO. Good old fashioned blocking sockets and threads will do you.

With NIO, you can register interest in OP_WRITE for the selection key, and you will get notified when there is room to write more data.

Answer 4

There are a few things you need to do, assuming you already have a loop using Selector.select(); to determine which sockets are ready for I/O.

• Set the socket channel to non-blocking after you've created it, sc.configureBlocking(false);
• Write (possibly parts of) the buffer and check if there's anything left. The buffer itself takes care of current position and how much is left.

Something like

sc.write(bb);
if(sc.remaining() == 0)
   //we're done with this buffer, remove it from the select set if there's nothing else to send.
else
    //do other stuff/return to select loop

• Get rid of your while loop that sleeps

Answer 5

I am facing some of the same issues right now:
- If you have a small amount of connections, but with large transfers, I would just create a threadpool, and let the writes block for the writer threads.
- If you have a lot of connections then you could use full Java NIO, and register OP_WRITE on your accept()ed sockets, and then wait for the selector to come in.

The Orielly Java NIO book has all this.
Also: http://www.exampledepot.com/egs/java.nio/NbServer.html?l=rel

Some research online has led me to believe NIO is pretty overkill unless you have a lot of incoming connections. Otherwise, if its just a few large transfers - then just use a write thread. It will probably have quicker response. A number of people have issues with NIO not repsonding as quick as they want. Since your write thread is on its own blocking it wont hurt you.

Answer 6

I second Toms answer about registering OP_WRITE on the selection key.

I would recommend the Rox Java NIO Tutorial.

Answer 7

The more I read about Java NIO, the more it gives me the willies. Anyway, I think this article answers your problem...

http://weblogs.java.net/blog/2006/05/30/tricks-and-tips-nio-part-i-why-you-must-handle-opwrite

It sounds like this guy has a more elegant solution than the sleep loop.

Also I'm fast coming to the conclusion that using Java NIO by itself is too dangerous. Where I can, I think I'll probably use Apache MINA which provides a nice abstraction above Java NIO and its little 'surprises'.