How to decide on what hardware to deploy web application

Question

Suppose you have a web application, no specific stack (Java/.NET/LAMP/Django/Rails, all good).

How would you decide on which hardware to deploy it? What rules of thumb exist when determining how many machines you need?

How would you formulate parameters such as concurrent users, simultaneous connections, daily hits and DB read/write ratio to a decision on how much, and which, hardware you need?

Any resources on this issue would be very helpful...

Specifically - any hard numbers from real world experience and case studies would be great.

Answer 1

Determine your expected load. Setup a machine and run some tests against it with a Load testing tool. How close are you if you only accomplished 10% of the peak load with some margin for error then you know you are going to need some load balancing. Design and implement a solution and test again. Make sure you solution is flexible enough to scale.

Trial and error is pretty much the way to go. It really depends on the individual app and usage patterns.

Answer 2

Test your app with a sample load and measure performance and load metrics. DB queries, disk hits, latency, whatever.

Then get an estimate of the expected load when deployed (go ask the domain expert) (you have to consider average load AND spikes).

Multiply the two and add some just to be sure. That's a really rough idea of what you need.

Then implement it, keeping in mind you usually won't scale linearly and you probably won't get the expected load ;)

Answer 3

Capacity Planning is quite a detailed and extensive area. You'll need to accept an iterative model with a "Theoretical Baseline > Load Testing > Tuning & Optimizing" approach.

Theory

The first step is to decide on the Business requirements: how many users are expected for peak usage ? Remember - these numbers are usually inaccurate by some margin.

As an example, let's assume that all the peak traffic (at worst case) will be over 4 hours of the day. So if the website expects 100K hits per day, we dont divide that over 24 hours, but over 4 hours instead. So my site now needs to support a peak traffic of 25K hits per hour.

This breaks down to 417 hits per minute, or 7 hits per second. This is on the front end alone.

Add to this the number of internal transactions such as database operations, any file i/o per user, any batch jobs which might run within the system, reports etc. Tally all these up to get the number of transactions per second, per minute etc that your system needs to support.

This gets further complicated when you have requirements such as "Avg response time must be 3 seconds etc" which means you have to figure in network latency / firewall / proxy etc

Finally - when it comes to choosing hardware, check out the published datasheets from each manufacturer such as Sun, HP, IBM, Windows etc. These detail the maximum transactions per second under test conditions. We usually accept 50% of those peaks under real conditions :)

But ultimately the choice of the hardware is usually a commercial decision.

Also you need to keep a minimum of 2 servers at each tier : web / app / even db for failover clustering.

Load testing

It's recommended to have a separate reference testing environment throughout the project lifecycle and post-launch so you can come back to run dedicated performance tests on the app. Scale this to be a smaller version of production, so if Prod has 4 servers and Ref has 1, then you test for 25% of the peak transactions etc.

Tuning & Optimizing

Too often, people throw some expensive hardware together and expect it all to work beautifully. You'll need to tune the hardware and OS for various parameters such as TCP timeouts etc - these are published by the software vendors, and these have to be done once the software are finalized. Set these tuning params on the Ref env, test and then decide which ones you need to carry over to Production.