Critique my server design please.

Question

Hi Guys,

I am designing a server in java which would be used to trade bonds. This server will act as a mediator between the client UI and the analytical server. The analytical server is the brain, my server will simply interact with it (using tcp sockets) and forward the responses to the client.

The server is expected to handle ~500 clients concurrently. And it has to be scalable and efficient to handle ~500 messages per second.

The flow of messages between the client UI and the analytical server is this:

1. Client through the UI requests for price.
2. My server accepts the message, formats it and then asynchronously
   sends to the analytical server.
3. When the response from the analytical server arrives, format and send
   the response to the client UI.
4. If the client likes the price, he/she will request to trade.
5. Repeat steps 2 & 3.

There is an existing framework that I leverage to handle authentication and to send messages between my server and the client UI. I have already coded the messaging bit between my server and the analytical server.

So as far as designing the remainder of my server, I am thinking of having 4 blocking queues. When the request for price comes, I immediately insert the request in a queue (Queue1). I then have a processor which takes messages out of Queue1, formats it and puts it in another queue (Queue2). The processor class internally contains a thread pool (Executors.fixedThreadpool) and formatting of each message occurs in a separate thread.

I then have a dispatcher class which contains another thread pool. This dispatcher class is responsible for taking messages from Queue2 and writing it to the Analytical server.

When I receive the message from the analytical server, I insert it into another queue (Queue3). I have another thread pool which dequeues and formats the message and puts it into another (Queue4). And finally I have another thread pool with pops messages out of Queue4 and publishes to the client.

I reason I chose 4 different queues is because if I want, I can write a tool to observe the size of these queues. The information about the size of these queues will

1. Allow me to tune the size of the thread pools. 
2. Further, if needed, I can publish the messages contained in these queues
   thus allowing me to monitor all the messages that flows through my server.

So what do you guys think? Any ideas, tips are most welcome.

Cheers

Answer 1

So it sounds like you'll end up with 500 threads, if it takes longer than 1 second to process the message?

Answer 2

First of all, when you say your server must "handle" ~500 messages per second, what exactly do you mean by "handle"? Accept? Because if you mean anything else, I'm not sure how that jibes with analytical server being asynchronous.

Secondly, I think you have 3 queues too many :-) You need a queue to act as sync-to-async buffers between your server and the analytical server. Beyond that, there's little point in queueing anything else (well, that depends on how exactly you're getting a response back from the analytical server; if for some reason you're polling it instead of being notified then you may need the queue there as well).