What tool(s) for fairly complex stress testing a web application

Question

I'm part of a team developing a new web-based product for our company. Whilst we're pretty positive the application will work perfectly with our expected average of 10-50 concurrent users, we have a requirement to ensure it will continue to work correctly for 1000 concurrent users.

Our primary concern is around the loading and displaying of various documents from the file system. To get the files, the user would need to log in, navigate to a search page, wait for a search to complete, then click a link and wait for the document to load.

We have SilkTest/TestPartner, which can be configured to go through the steps above, but I don't believe these tools can realistically be used to simulate load to the extent we require.

Can you suggest some (preferably free / open source) programs? Commercial products aren't a no-go, but the price cannot be enormous :)

Thanks in advance!

Answer 1

Use apache's benchmark tool ab.

From Wikipedia: ApacheBench:

ApacheBench is a command line computer program for measuring the performance of HTTP web servers, in particular the Apache HTTP Server. It was designed to give an idea of the performance that a given Apache installation can provide. In particular, it shows how many requests per second the server is capable of serving.

You can provide it with cookie/value pairs of a logged in user, and point it to the search URL, with the ability to tell it to run concurrent connections to simulate concurrent users.

Answer 2

There's also httperf:

http://agiletesting.blogspot.com/2005/04/http-performance-testing-with-httperf.html

Answer 3

I recommend you to try the Microsoft Web Capacity Analysis Tool (WCAT). It is able to run on multiple machines simultaneously, and each of the clients can have multiple "virtual clients", thus they are able to simulate multiple connections to a web server. It can connect to any web server (used by some Apache fanatics), but of course it integrates best with IIS.

It was developed by the IIS performance team and is used by Microsoft to stress-test their IIS server. It has both x86 and x64 versions, and it is easy to create your own extensions to it.

It works the following way: you create a scenario file with which you can describe any number of HTTP requests. (It has options for GET, POST, and also has HTTPS support.) You can then group these requests to transactions which represent a series of requests to simulate user interaction. Each virtual client runs a single transaction at a time, and it waits until the current request completed before starting the next one.
It is easy to write a transaction that contains a series of requests which can simulate users "clicking" on things. (It is also possible to "sleep" for a certain amount of time between requests, thus simulating a user reading the page.)

Here is the official introduction:

Web Capacity Analysis Tool (WCAT) is a lightweight HTTP load generation tool primarily designed to measure the performance of a web server within a controlled environment. WCAT can simulate thousands of concurrent users making requests to a single web site or multiple web sites. The WCAT engine uses a simple script to define the set of HTTP requests to be played back to the web server. Extensibility is provided through plug-in DLLs and a standard, simple API.

Perhaps its only downside is that it is a bit compicated to set up at first. It is a command line tool so it is also a bit too easy to screw up with the parameters.

Answer 4

the grinder for java apps : Grinder

Answer 5

Apache JMeter would definitely support your scenario and supports Distributed Testing (pdf) which is a mode where you run several agents on different machines and can thus scale to simulate high numbers of concurrent users.

Answer 6

You could try Web Performance Load Tester - it is a mid-priced load testing product. It is much less expensive than SilkPerformer or LoadRunner and much more productive. It is limited to only web protocols, but that doesn't sound like it will be a problem for your application. There are videos on the site that will give you a pretty good feel for the tool: http://www.webperformanceinc.com/load_testing/demo/

(disclaimer: I work for Web Performance and use the product nearly every day)