Stateless Blocking Server Design

Question

Hello Ladies and Gents,

A little help please.

I am designing a stateless server that will have the following functionality:

1. Client submits a job to the server.
2. Client is blocked while the server tries to perform the job.
3. The server will spawn one or multiple threads to perform the job.
4. The job either finishes, times out or fails.
5. The appropriate response (based on the outcome) is created, the client is unblocked and the response is handed off to the client.

Here is what I have thought of so far.

1. Client submits a job to the server.
2. The server assigns an ID to the job, places the job on a Queue and then places the Client on an another queue (where it will be blocked).
3. Have a thread pool that will execute the job, fetch the result and appropriately create the response.
4. Based on ID, pick the client out of the queue (thereby unblocking it), give it the response and send it off.

Steps 1,3,4 seems quite straight forward however any ideas about how to put the client in a queue and then block it. Also, any pointers that would help me design this puppy would be appreciated.

Cheers

Answer 1

How are you communicating to the client?

I recommend you create an object to represent each job which holds job parameters and the socket (or other communication mechanism) to reach the client. The thread pool will then send the response to unblock the client at the end of job processing.

Answer 2

Why do you need to block the client? Seems like it would be easier to return (almost) immediately (after performing initial validation, if any) and give client a unique ID for a given job. Client would then be able to either poll using said ID or, perhaps, provide a callback.

Blocking means you're holding on to a socket which obviously limits the upper number of clients you can serve simultaneously. If that's not a concern for your scenario and you absolutely need to block (perhaps you have no control over client code and can't make them poll?), there's little sense in spawning threads to perform the job unless you can actually separate it into parallel tasks. The only "queue" in that case would be the one held by common thread pool. The workflow would basically be:

1. Create a thread pool (such as ThreadPoolExecutor)
2. For each client request:
   1. If you have any parts of the job that you can execute in parallel, delegate them to the pool.
   2. And / or do them in the current thread.
   3. Wait until pooled job parts complete (if applicable).
   4. Return results to client.
3. Shutdown the thread pool.

No IDs are needed per se; though you may need to use some sort of latch for 2.1 / 2.3 above.

Timeouts may be a tad tricky. If you need them to be more or less precise you'll have to keep your main thread (the one that received client request) free from work and have it signal submitted job parts (by flipping a flag) when timeout is reached and return immediately. You'll have to check said flag periodically and terminate your execution once it's flipped; pool will then reclaim the thread.

Answer 3

The timeouts will be somewhat tricky, and will have hidden gotcha's but the basic design would seem to be to straightforward, write a class that takes a Socket in the constructor. on socket.accept we just do a new socket processing instantiation, with great foresight and planning on scalability or if this is a bench-test-experiment, then the socket processing class just goes to the data processing stuff and when it returns you have some sort of boolean or numeric for the state or something, handy place for null btw, and ether writes the success to the Output Stream from the socket or informs client of a timeout or whatever your business needs are

If you have to have a scalable, effective design for long-running heavy-haulers, go directly to nio ... hand coded one-off solutions like I describe probably won't scale well but would provide fundamental conceptualizing basis for an nio design of code-correct work.

( sorry folks, I think directly in code - design patterns are then applied to the code after it is working. What does not hold up gets reworked then, not before )