Comparing scalable web-app architecture on Java and .NET

Question

What are the recommended steps for creating scalable web/enterprise applications in Java and .NET? I'm more interested in what it takes to go from a low volume app to a high volume one (assuming no major faults in the original architecture). For example, what are the options in each platform for clustering, load-balancing, session management, caching, and so on.

Answer 1

Just came across this explanation of how MySpace scaled up, really interesting and exactly the kind of information I'm looking for. However, there is no discussion about the technologies used to create the partitioning or caching and so on...

Answer 2

Perhaps there is so many options, I am not sure anyone to considered all options in full. Usually, the choice of language comes first and then you decide how to achieve the results you want.

My guess is you have good options in just about any major general purpose language. This would have been an issue 10 years ago, but not today IMHO.

A good comparison of available open source technologies in Java is http://java-sources.org/ As you drill down into each category, you can see this is fairly extensive and it does even consider commercial products or C#

For any scalable system, you need to consider hardware solutions as well, such as network components.

My suggestion for turning a low volume app into a high volume one, is start with something fairly high volume in the first place and expect to re-engineer it as you go.

What sort of volumes are you talking about? Everyone has different expectations of high and low volume. e.g. When google have lower than average volumes they switch of whole data centres to save power. ;)

Answer 3

Short answer - it depends. What (if any) business goals are involved? What is the project budget and timeframe? Which industry (is it regulated)?

Answer 4

Unfortunately there are a lot of aspects of your question that are going to be context dependant. Scalability can mean different things depending on your requirements and the application. The questions that need answering as you come up with your architecture (regardless of the technology you are using) include:

• For example do you want to support users working with very large amounts of data or do you simply need to support a large number users each working with relative modest amounts of data?
• Does your application do more reading than writing, since reads are cheap and writes are expensive a read heavy application can be simpler to scale than a write heavy one.
• Does the data in your application have be always consistent or will eventual consistency suffice? Think posting a message to a social networking site versus withdrawing money from a bank account).
• How available does you application need to be? Strict high availability requirements will probably require you to have smooth fail-over to other servers when one server crashes.

There are many other questions like this that need to be answered about your specific application before a proper discussion on your application architecture can take place.

However so I don't just leave you questions and not a single answer here is what I think is the single most important thing to take into account when designing a web application for scalability: Reduce (down to zero if possible) the amount of shared session state in your application (global application counters, cached primary key blocks and the like). Replicating shared state across a cluster has a very high cost, when you are trying to scale up