how to scale this single threaded java client?

Question

I'm writing a Stomp protocol client with Java and it has only one thread to process IO. That means that thread reads and writes incoming data to the application back and forth. My issue is if I need to scale this application in future with multi threading and NIO, how that could be arranged?

my IO processor thread is called "TcpLink" link and it has following skeletion

    class TcpLink implements Runnable {

         public void run() {
            // read data from socket and assign it to  a byte buffer
            // notify the listening application
         }
   }

If I need to allow multiple threads to dispatch the incoming messages, how this class should be changed?

thanks!

Answer 1

I really like this Doug Lee presentation about designing scaleable systems based around Java NIO.

In essence your design will typically be based around the Reactor pattern whereby a single I/O thread round-robins over a number of client connections. If the I/O thread ever becomes saturated you could consider farming off connections from a master reactor to one or more child reactors; each reactor containing its own thread.

Another important point to note in your design is that the I/O thread should only perform I/O and should typically dispatch any inbound messages to a separate thread (e.g. ExecutorService) to do any actual work. This prevents other connections from being starved of I/O whilst a given message is being processed.

Answer 2

It will be a better idea to write a wrapper class which manages a list of TcpLink classes. The role of the wrapper class will be receiving the incoming messages and assign it to a waiting/feww TcpLink instance, you can calculate the throughput/TPS and create new instances when ever you need. For now you will create only one of it so that it is a single thereaded.