What are some good resources for designing a web-services based architecture that can scale well?

Question

I have a web application that currently has about 500,000 users, averaging about 500 page views per hour, and we are expecting to have to scale up to 20,000,000 users within the next year.

Our current system cannot handle that scale, and so we need to move to one that can.

I was thinking a services-based architecture would be more robust and allow for the component services to be scaled independently of each other and the main application.

But I have been warned that one can easily design a services architecture that doesn't scale well, and since I have little to no experience designing such a system, going through some recommended resources is probably better than just "taking a stab at it."

So, what resources are recommended for architecting a well-scaling system? The platform is .Net, but I would imagine that the same information applies to any service-based architecture, regardless of platform.

Answer 1

Theoretically it scales, but you need to be careful of latency. Say you externalize a module, now you can have two services, serving in place of one, but in the time it takes to send, process, reply to one request on an external service you could have processed 100 internal requests, so unless you have 100 servers your actually getting lower throughput, but either way your paying much more in maintenance and space with 100 servers versus 1. Normally the balance has to be measured between latency and processing, as a rule of thumb, if you need to send lots of data, you might be better off not separating that step into a service. Luckily you have a running system against which you cab measure, most people have to guesstimate this from specs alone.

So consider your data flow patterns and favor locality, it's an art to cut your app onto services along the grain, and not against it.

You might be interested in reading about Concurrent patterns, as that is what it boils down to.

Answer 2

I'd first answer Gennady Shumakher's comment - "What is currently your system's bottle neck you are considering as not scalable?". Once you know that, then you can approach it specifically, then the whole application as you need to.

One interesting study is Twitter. It began with only small numbers of requests, but has quickly grown into a heavily used platform, with world-wide traffic. This may be useful. Also interesting might be this blog on scalability.

Answer 3

I'd like to recommend Ebay architectural principles presentation and also a good writeup on the presentation. Actually Ebay adopted a typical service oriented approach. Services are extracted by functions so that each service can scale independently. I think there are some good points you can learn from.