*Client* scalability for large numbers of remote web service calls

Question

Hey Guys,

I was wondering if you could share best practices and common mistakes when it comes to making large numbers of time-sensitive web service calls.

In my case, I have a SOAP and an XML-RPC based web service to which I'm constantly making calls. I predict that this will soon become an issue as the number of calls per second will grow.

On a higher level, I was thinking of batching those calls and submitting those to the web services every 100 ms. Could you share what else works?

On a lower level side of the things, I use Apache Xml-Rpc client and standard javax.xml.soap.* packages for my client implementations. Are you aware of any client scalability related tricks/tips/warnings with these packages?

Thanks in advance

Yuriy

Answer 1

One thing to realize is that in JavaScript you're always dealing with an event pump: some browser event happens, or a timer expires, and then a chunk of JavaScript gets executed. With that execution model in mind, you don't really want to think of having a periodic batch and send process going on - what you want to do instead is batch together all the calls that happen in one event pump (the chunk of javascript executing in response to one event from the browser) and send them off.

This is done by changing your rpc code so that every call queues up calls with parameters and callback functions into a global array, and if it's queueing up the first call it also schedules with setTimeout(..., 0) a function that will send off everything in the queue and clear the array.

Start with that and then you can experiment later with other execution models, like firing off the first request right when you get it and sending everything else in a batch after the current event pump is done.