What is a good standard metric for stability in a web application?

Question

We currently facing some stability issues with our develop web application product. This product was built in part by our partner contractor, and we want to have a good standard metric for stability. The issues we have been facing is constant crashing. The web application is unable to identify when there are more request than it can handle, it builds up memory (like a memory leak) and later it dies without any type of possible recovery.

We would like to write a very simple measurement for our partner contractor to meet. We thought about a few ideas:

• A system that is able to identify high loads of request and serve server unavailable try again pages, until it recovers from the high load.
• A set number of users concurrent or pageviews that will allow us to have a clear metric of when to use scalability options like Load Balancer and Caching.

At this moment we have to use caching and load balancing to be able to recycle the web applications every x hours (depending on the load) so they don't die constantly.

Thanks for your help.

Answer 1

"High load" is really hard to define.

It's much, much easier for you to determine what the minimally acceptable service levels are.

1. Minimum number of concurrent requests.

2. Maximum time to serve a request.

3. Minimum number of requests per hour.

Simple service levels like that are easy to measure, easy to simulate and easy to write into a contract. Any lawyer or accountant can review the results of a load test and say they did or did not meet the minimums. No deep technical knowledge needed at all.

Note that when you do this, "minimums become maximums". If you say they must serve a minimum of 10,000 requests per hour, your testing will often reveal that's the maximum, also.

So define your minimums and maximums from your business model. How many do you need to keep people happy and productive? Asking for more is silly. Asking for fewer means unhappy or unproductive users.

Answer 2

Typically, I've seen these performance type requirements built into the specification... "the system should support x amount of concurrent users" or "x number of requests per hour".

These things can be easily tested and verified using something like LoadRunner or you can roll your own similar load tester using something like HttpUnit.

Answer 3

What you'd need are following parameters:

1. Load Test:

   i) Estimate your application usage behaviour.. (i.e.) no. of concurrent users expected, typical user activity

   ii) Load the application gradually and look at parameters like cpu usage, response times, throughput, etc.

2. Sustainability:

   Load the application (at optimum load) for a considerable amount of time (12-24 hrs) and look at the same parameters cpu usage, error levels, etc.

You can also try scalability, where you would add hardware incrementally and monitor the behaviour of the application.

This should give you a decent understanding of the behaviour of your system.