load-testing a webapp

Question

Good folks of SO,

When load testing a basic web application, what sanity checks do you do other than expected response time?

Is it fair to ask for peak memory usage? what other checks do you make?

thank you

Answer 1

On the server

• Requests per second the application can withstand
• Requests per second that hit the database (if any, related to the number above, but it's useful to have them as separate figures)
• Transferred bandwidth (separated by media type, if possible)
• CPU utilization
• Memory utilization

On the client

• Response time
• Weight of the average page
• Is the CPU usage high at any time
• Run something like YSlow to see what can you optimize on the output to make it quick for users

Stress testing tools usually come with most of these measures (except for Memory, CPU and database usage), as do YSlow or Firebug do on the client.

Answer 2

We look at a pretty wide variety of metrics when analyzing the results of a load test.

On the server, we start with these main 4 categories:

• CPU (% utilization, context switches/sec, process queue length)
• Memory (% use, page reads/sec, page writes/sec)
• Bandwidth (incoming, outgoing, send & receive errors, # connections, connection failures, segment retransmits/sec)
• Disk (Disk I/O Time %, avg service time, queue length, reads and writes/sec)

We also like look at metrics specific to the webserver and application server in use. For example, in IIS we look at IIS connection counts, cache hit rates and turnover frequency, etc. In .NET, we would be looking at ASP.NET Requests/sec, ASP.NET Last Request Execution Time, ASP.NET Current Requests, ASP.NET Queued Requests, ASP.NET Request Wait Time, ASP.NET Errors/sec and many others.

On the client side, we are primarily looking at total load time for the pages, duration and TTFB (time to first byte) for critical transactions, bandwidth usage, average page size and failure rate. We also find two metrics very useful - we call them Waiting Users and Average Wait Time. Not many tools have these - they tell you at each sample period exactly how many simulated users are in the process of retrieving a resource from the server and how long, on average, they have been waiting for the resource to arrive. We find these very useful for

• determining when the server has reached its capacity
• discovering that the server has stopped responding to certain types of requests (typically for certain resources, such as those requiring a database query)