RFC with implementing a modular architecture

Question

Looking for opinions on the modularization of web applications. Already most applications regardless of language have a backend DB and support tie-ins with their respective web application server (Apache, IIS, Lighttp, etc ) but a lot of developers I've dealt with have problems coming to terms with using Memcached or anything outside of the web app's immediate process space.

Is modularization of a web application as good thing a thing as I believe or is there something I'm missing that causes everyone from Sr. Developers to CTO's to be hesitant to move specific parts of the business logic out of the web front end and into specialized backend services?

For example, a few years ago I got shot down in a project design meeting for a very high traffic website when I suggested we rip the process intensive ACL logic out of the front end framework and turn it into a semi-clusterable service application in the backend. For me the benefits was a cleaner separation of code and the ability to reuse the ACL logic in multiple places by using REST/JSON as the bridge between say PHP & Python.

The developers who disagreed with my idea argued it was "too complicated" but I just couldn't see how? My arguments are that just as there can be tag soup for the presentation layer, there can and is often a logic soup of code that's so meshed together that if a issue arises it might be nigh on impossible to perform a "surgical" fix.

So to shorten it down, what are the con's & or pro's of break large applications down into independent but cooperative processes ( not threads or sub requests ). MySQL, Memcache, similar service process are great...but why not anything else? How is going down this path "too complicated"?

Answer 1

Well, sometimes "too complicated" means "I don't want to think outside my confort zone."

The basic notion sounds good --- you're talking about pretty plausible service-oriented architecture here.

Now, as far as the pros and cons, the first thing against it is that you do indeed have to get people to think outside their comfort zones. A more technical con is that you need to preserve what is, in effect, the session state. Say you pick up the authentication token from your auth service; how is that token going to stay associated with the correct user session.

Another issue is that it might be harder to debug, since it's happening more dynamically.

On the pro side, though, if you can satisfy the session-state issue, you get a highly scaleable architecture; if you need more service, you can either enlarge the server or simply add another server.

Answer 2

I am a fan of separating the core server/business logic functionality from the web application code. This is for a few different reasons:

1. You can better control the business logic. There will be no way to mix this in with GUI code and create a mess. You want to keep all the business logic separate so you can later make an API to call the code functionality and not hope that no business logic made its way into the GUI code.
2. Right from the get-go you have to design a good API that you have to use yourself to communicate with your server from the client. The client can be the web interface or it can be a remote user.
3. Stability. GUI web code can easily be written wrong and consume too much memory thereby taking down the entire app.
4. To scale you can move these separate components to different servers. If you are using a caching system that already allows for clustering scaling up can be as simple as just adding more caching servers.
5. I've found that application updates are made most often to the GUI code so the core service does not have to be taken down most of the time.
6. Security. You can be sure that the security code is implemented in the server code so if someone uses your API they won't be able to bypass any security code that made its way into the GUI code.

Our system has a core 'engine' service implemented as a Java application. The web application is also written in Java and (currently) communication is via RMI. Our site/application is growing and we haven't yet had to start using a caching service like memcached or JCS.