What performance testing methodology are you using for your webapps?

Question

I would like to performance test a typical web application. The application offers some web2.0 functionalities, like writing blogs, wikis, search for contents and something like this. I've analysed the access log and got know an unterstanding what the users are doing really frequently.

The gap in my brain is how to proceed? I thoght about the following methodology:

• (A) split the functionality into transaction (write blogpost, view wikipage etc)
• (B) run these transactions with increasing amount of users
• (C) make some reports: "viewing wikipage with hardware x could be performed by y users concurrently, while memory is the bounding ressource"
• (D) try to mix multiple transactions into ONE scenario, which should be a realistic mapping of the real user load
• (E) run this scenario with increasing users, making the same reports as in C

What do you think about this and whats you methodology?

Answer 1

The problem you're going to run into is how to test all that reproducibly. Tests that aren't reproducible (i.e. manual tests) are of strictly limited utility.

Take a look at watir (pronounced like "water"); it gives good coverage and is very scriptable.

Answer 2

You first need to know which activities can be performed on your site, and more importantly need some idea of what proportion of your total traffic that activity takes up. For a simple blog you might say it looks like this:

• reading index page : 30%
• reading post pages: 65%
• creating comments: 4%
• creating posts: 1%

You can then use some sort of testing framework to simulate this load and understand how many requests per minute second you can sustain. This will give you some hard number on capacity. You can also profile your memory/cpu/network/whatever to see how they are utilised during this time.

However it really is important not to skip on the actual usability testing. In standard dynamic website thinsg will start to feel slow if it takes more than about half a seond to load a page. In an AJAX enabled site you will find that the increased amount of feedback available to the use gives them a higher tolerance for latency and the limits on what is acceptable will need to be investigated by human in order to tell.