How to load test a SOAP interface (are SOAP function calls atomic)?

Question

A server offers a SOAP function call interface.
Hundreds, perhaps a few thousand, of PCs call that function.
I need to load test this using a single PC.

At the moment I just have a for loop. Is that good enough? If not how can I make the test more realistic?

I had thought of threading, but if the SOAP function call is atomic then that doesn't buy anything. Even if it does, there can still only be one active thread (per CPU) on my tester PC.

I hope that is clear, please ask for further info if not.

What's the best way to test this, simulating many PCs on one?

---

Update, I am using VB Express 2008. I can't believe that .NET is so constructed that it will block the CPU for the duration of a SOAP Remote Procedure Call ... or does it?

Answer 1

The atomicity of the SOAP call has little impact on the usefulness of threading.

You should create as many threads as your system will allow before it slows down your test machine. Even more threads than CPUs will still generate a lot of load because the SOAP call will likely not return immediately. If you have 4 CPUs, the first 4 threads may simultanously make a SOAP call, then the next 4 threads can make another 4 calls before the original 4 are complete. While this does not exactly simulate multiple machine callers, you wont be able to exactly simulate that scenario without a set of test machines and a set of scripts to call the service at once.

EDIT: Fixed typo above, word should have been "likely" instead of "like"

Answer 2

Using a for loop, you are probably only issuing one simultaneous request. The thread running the loop will likely be blocked, waiting for the response. Note that this blocks the thread, but not the entire CPU, which can be shared by many threads. Therefore, by using multiple threads you can issue multiple requests simultaneously.

Now, the SOAP handler on the server could be single-threaded - which would mean that it can only handle 1 request simultaneously. This is much less likely - as server frameworks are specifically designed to not work this way. Hopefully you'll not discover this is the case.

Have you determined how many simultaneous requests you need to handle? Depending on the scenario, even a collection of 10,000 active clients may only be generating 10 simultaneous requests. This will depend on the frequency of the calls as well as the response duration. The request rate (requests/sec) will determine if you will be able to reliably/accurately simulate the desired load from a single computer.