Pattern for designing a scalable web service

Question

I am writing a web service in Java which needs to handle a large number of requests / second. The general flow will be:

• Web service receives a request from client
• Returns a 'keep polling me' response to client
• Calls another web service (or services), and waits for them to respond (with a timeout)
• Client polls our web service, until it receives a response (with a timeout)

Researching on the Internet, I have found two general approaches to writing web services:

• Spawn a thread for each request
• Use the Reactor pattern (central dispatcher thread responds to IO events)

Do you have a recommendation for which approach is generally better, and what are the pros/cons of each approach? I would also appreciate pointers to examples.

Answer 1

Don't think multi-threading. Think asynchronously. I happened to have just coded an async handler that ran 2,000 RPS with <10 threads in IIS. Not sure how java works since I'm a .net guy but I gotta believe they have similar BeginXXX/EndXXX methods. If you ever spawn a thread then you're not considering all the places your code can block: data base IO, File I/O, web services, etc. These are the places your performance will cause your site to be slow.

Async, Async, Async.

Chant and repeat.

Answer 2

In addition to "No Refunds No Returns" response, I'd say yeah "Think Asynchronously" as you should be allowing your container to manage the multi-threading/scalability and high-availability issues of the web services it has deployed, this allows you to set-up things like clustering and so forth using your application container.

EDIT: So in conclusion, there isn't a pattern as such, maybe you should explore the scalability/availability features of your application container...

Answer 3

Asynchronism is indeed the right approach but don't manage this yourself, use something that supports asynchronous web service invocation like JAX-WS 2.0 (which uses the Future interface and/or the Executor framework from java.util.concurrent). See Asynchronous Web Service Invocation with JAX-WS 2.0.