views:

136

answers:

5

What are some good automated tools for load testing (stress testing) web applications, that do not use record and replay of HTTP network packets?

I am aware that there are numerous load testing tools on the market that record and replay HTTP network packets. But these are unsuitable for my purpose, because of this:

  • The HTTP packet format changes very often in our application (e.g. when we optimize an AJAX call). We do not want to adapt all test scripts just because there is a slight change in HTTP packet format.

  • Our test team shall not need to know any internals about our application to write their test scripts. A tool that replays HTTP packets, however, requires the team to know the format of HTTP requests and responses, such that they can adapt details of the replayed HTTP packets (e.g. user name).

The automated load testing tool I am looking for should be able to let the test team write "black box" test scripts such as:

  • Invoke web page at URL http://... .
  • First, enter XXX into text field XXX.
  • Then, press button XXX.
  • Wait until response has been received from web server.
  • Verify that text field XXX now contains the text XXX.

The tool should be able to simulate up to several 1000 users, and it should be compatible with web applications using ASP.NET and AJAX.

A: 

A license can be expensive for it (if you dont have MSDN), but Visual Studio 2010 Ultimate edition has a great set of load and stress testing tools that do what you describe. You can try it out for free for 90 days here.

thekaido
I have already checked that out; Visual Studio Load Test, however, uses the HTTP record and reply approach, which I want to avoid.
fmunkert
A: 

There are a few approaches; I've been in situations, however, where I've had to roll my own load generating utilities.

As far as your test script is concerned it involves:

  • sending a GET request to http://form entry page (only checking if a 200 response is given)
  • sending a POST request to http://form submit page with pre-generated key/value pairs for text XXX and performing a regexp check on the response

Unless your web page is complex AJAX there is no need to "simulate a button press" - this is taken care of by the POST request.

Given that your test consists of just a 2-step process there should be several automated load packages that could do this.

I've previously used httperf for load testing a large website: it can simulate a session consisting of several requests and can simulate a large number of users (i.e. sessions) simultaneously. For example, if your website generated a session cookie from the home page you could make that the first request, httperf would then use that cookie for subsequent requests, until it had finished doing the list of requests supplied.

PP
Actually, I want to avoid sending GETs/POSTs at the HTTP level (see my question).
fmunkert
Your page is a complex AJAX page. You should know that there are no programming languages that can emulate a javascript browser. Your best bet is installing several computers running lots of instances of your favourite browser controlled by a custom written script that communicates with your favourite browser. Naturally this is a huge amount of work and so you will find most pragmatic system administrators use a more simplified approach to load testing.
PP
Actually, running 1000 PCs per user in Amazon's Elastic Compute Cloud is one thing we already have thought about. The price would be OK for us, but we would prefer an in-house solution. A load testing tool that emulates a JavaScript browser is exactly what I am looking for. I think it cannot be that hard to implement such a tool, because the tool basically only needs to support JavaScript (e.g. use Microsoft's free scripting engine) and understand the DOM. All the complex parts such as CSS or rendering would not be required in such a load testing tool.
fmunkert
A: 

JMeter I've found to be pretty helpful, it also has a recording functionality to record use cases so you don't have to specify each GET/POST manually but rather "click" the use case once and then let JMeter repeat it.
http://jakarta.apache.org/jmeter/

nillls
Unfortunately, this tool also falls into the HTTP record+replay category, which I explicitly want to avoid.
fmunkert
+1  A: 

TestMaker by PushToTest.com can run recorded scripts such as Selenium as well as many different languages like HTML, Java, Ruby, Groovy, .Net, VB, PHP, etc. It has a common reporting infrastructure and you can create load in your test lab or using cloud testing environments like EC2 for virtual test labs.

They provide free webinars on using open source testing tools on a monthly basis and there is one next Tuesday.

http://www.pushtotest.com

Troy a.
+1  A: 

What about http://watin.sourceforge.net/ ?

Tomas Voracek