What are the theoretical performance limits on web servers?

Question

In a currently deployed web server, what are the typical limits on its performance?

I believe a meaningful answer would be one of 100, 1,000, 10,000, 100,000 or 1,000,000 requests/second, but which is true today? Which was true 5 years ago? Which might we expect in 5 years? (ie, how do trends in bandwidth, disk performance, CPU performance, etc. impact the answer)

If it is material, the fact that HTTP over TCP is the access protocol should be considered. OS, server language, and filesystem effects should be assumed to be best-of-breed.

Assume that the disk contains many small unique files that are statically served. I'm intending to eliminate the effect of memory caches, and that CPU time is mainly used to assemble the network/protocol information. These assumptions are intended to bias the answer towards 'worst case' estimates where a request requires some bandwidth, some cpu time and a disk access.

I'm only looking for something accurate to an order of magnitude or so.

Answer 1

I think there are too many variables here to answer your question.

What processor, what speed, what cache, what chipset, what disk interface, what spindle speed, what network card, how configured, the list is huge. I think you need to approach the problem from the other side...

"This is what I want to do and achieve, what do I need to do it?"

Answer 2

Read http://www.kegel.com/c10k.html. You might also read StackOverflow questions tagged 'c10k'. C10K stands for 10'000 simultaneous clients.

Long story short -- principally, the limit is neither bandwidth, nor CPU. It's concurrency.

Answer 3

This will depend what is your CPU core What speed are your disks What is a 'fat' 'medium' sized hosting companies pipe. What is the web server?

The question is too general

Deploy you server test it using tools like http://jakarta.apache.org/jmeter/ and see how you get on.

Answer 4

OS, server language, and filesystem effects are the variables here. If you take them out, then you're left with a no-overhead TCP socket.

At that point it's not really a question of performance of the server, but of the network. With a no-overhead TCP socket your limit that you will hit will most likely be at the firewall or your network switches with how many connections can be handled concurrently.

Answer 5

I think it really depends on what you are serving.

If you're serving web applications that dynamically render html, CPU is what is consumed most.

If you are serving up a relatively small number of static items lots and lots of times, you'll probably run into bandwidth issues (since the static files themselves will probably find themselves in memory)

If you're serving up a large number of static items, you may run into disk limits first (seeking and reading files)

Answer 6

In any web application that uses a database you also open up a whole new range of optimisation needs.

indexes, query optimisation etc

For static files, does your application cache them in memory?

etc, etc, etc

Answer 7

If you are not able to cache your files in memory, then disk seek times will likely be the limiting factor and limit your performance to less than 1000 requests/second. This might improve when using solid state disks.

Answer 8

Six years ago, I saw an 8-proc Windows Server 2003 box serve 100,000 requests per second for static content. That box had 8 Gigabit Ethernet cards, each on a separate subnet. The limiting factor there was network bandwidth. There's no way you could serve that much content over the Internet, even with a truly enormous pipe.

In practice, for purely static content, even a modest box can saturate a network connection.

For dynamic content, there's no easy answer. It could be CPU utilization, disk I/O, backend database latency, not enough worker threads, too much context switching, ...

You have to measure your application to find out where your bottlenecks lie. It might be in the framework, it might be in your application logic. It probably changes as your workload changes.