Design Decision - Scaling out web based application's architecture

Question

This question is about design decision. I am currently working on a web project that will have 40K users to start with and in couple of month expected to grow 50M users (not concurrent users though). I would like to have a architecture that can be scaled out easily without much effort.

In order to explain, I would like to use a trivial scenario. Lets say, User entities and services such as CreateUser, AuthenticateUser etc., are a simple method calls for the Page Controllers. But once the traffic increases, for example, authenticating user (or such services related to user entities) has to be moved out to a different internal server to spread the load. But at the same time using RPC calls over the network when the user count is 40K would become overkill.

My proposal was to use IPC initially and when we need to scale out we can interally switch to TCP based RPC calls so that it can easily scale out. For example, I am referring to System.IO.Pipes.NamedPipeStreamServer to start with and move on to a TcpListener later on.

If we have proper design that can encapsulate above said approach, it would easy for us to scale out services into multiple network servers but at the same time avoid network calls when the user count is small.

Is this is a best approach? Any suggestions would be great ..

Note: The database scaling is definetly the second phase optimization so we have already made architectural design in place to easily partition data when traffic increases. The primary bottleneck would be application servers over the time period.

Answer 1

If what you're planning on doing is (if I read it right) farm out the authentication and authorization work to a central server, then I think you're going to find problems in scalability if you try and do it with named pipes or even low-level TCP sockets. There's no reason why you can't reach these internal servers over regular web services or even TCP-channel based WCF services.

The reason I would go this route is because invoking stateless web services (ASMX or WCF) is going to allow you to make your "auth and auth" (authentication and authorization) server as well as your user management server (createuser, etc) on a farm. So, as your hits to these services increase, you can scale out the number of servers responding to these calls without having to change the client code.

Answer 2

In my experience there's always a tension between "you don't need it now, so don't waste effort on it" and "here be dragons".

Your scaling strategy is, when the need arises, use particular remotng techniques to offload pieces of work to other hosts. Sounds like it might work. [In passing, another approach is just to have many parallel instances of the same thing, so keeping everything local - my instinct is that this might be better. But let's stick with your plan for now ...]

My general recommendation is to attack risk early. So in this case you are intending in the future to use a remoting technology to offload some work. Adding this new (to your system) technology will have (at least) two impacts:

1. New failure modes
2. Increased latency

Oh, and there's the (admitedly unlikely) possibility that the remoting strategy doesn't work! You may not see the scaling benefits you expect. Performance is notoriously unintuitive.

So I see risk here, I want to address this risk now. I would say do the remoting immediately even though it's not ncessary. You will then bake into all your performance testing the increased latency and into all your resilience testing the failure modes. You are doing this while the pressure is off, while the user population is low. You can also do some testing of the actual scalability.

Answer 3

In one of my project for travel industry (around 1m hits per day), we had a separate auth server farm. There were around four load balanced server at that time. Our business layer called the authentication web service (asmx) passing the user credentials and get the xml results. If the number of users increase, we can scale out the auth farm further. IMHO using the web services over the http (on intranet) give more performance benefit than TCP.