What are recommended strategies for load-testing (i. e. simulating large numbers of clients) server applications in a small shop?

Question

I develop a network based, multi-component software system that's designed to run on an arbitrary number of machines. I'm assuming a typical setup of 1 to 4 machines.

I want to be serious about testing the system, and I have set up a network of virtual machines on a strong PC that I can use to simulate the network interaction. However, that setup is not quite sufficient.

For example, because of the virtualization (I use qemu), each node always runs on one core only, so I can't test performance issues on code designed for multicore use. It would be nice if I could sometimes run any one of the virtual machines on the whole PC, to see what difference it makes, especially performance-wise, but also to check on some multithreading issues.

Buying more boxes and using harddisk partitions instead of virtual disk images are options, but are there more elegant approaches? I'm bootstrapping a company here and can't really afford loads of hardware yet and having to deal with real physical media for each machine rather than just moving image files around is certainly more work.

ETA: The application is a middleware system and does not really have a user interface. Testing is done through dummy clients and by feeding data into the system that's extracted again on the other end. It's not a website and it is usually not used over the Internet but rather inside the local network. UI and Internet interfaces are going to be provided at a later stage by some of the components that are hooked up to the system.

Answer 1

You can hire someone, maybe remotely, to perform the tests. And if you hire a remote worker, you will be able to test how well your application works over Internet

Answer 2

This topic has come up a few times on SO so far:

• What have you used to test your network service with its custom protocol?.
• Load test/stress testing web services
• Best way to stress test a website
• How do you test the performance of a website?

There are a number of tools out there to do network stress testing, but chances are that you will need to adapt them to fix your network configuration. For instance, I'm working on a project which does a lot of streaming stuff, so we had to write our own streaming client to test the network out.

Another thing is that you will likely need to set up a small network to test your production network -- network load testing tools also require a fair amount of CPU, so you usually need multiple machines running them simultaneously in order to really work the network. In our case, we have a 1:2 ratio -- that is, 1 testing client for each 2 servers we're going to test. You'll have to play around with things a bit to determine how many testing boxes you'll need, but just keep an eye on the performance of your testing machines so you can get a feel of how many you'll need to effectively benchmark your site.

Answer 3

I would suggest VMware Workstation which enables you to configure multiple processors for your VMs. A single license is extremely cheap, you could go with VMware Server (free) but you'll find Workstation to be more powerful in the short term I would think.

Additionally I would suggest a second PC, that is capable of booting from USB, that you can store your physical VM partitions on. One low-end dual core PC is not expensive especially if it only needs to boot into text mode.