Making decisions about the architecture of your project; what is your decision process like?

Question

Many of us, who have designed and developed systems from scratch, have been in situations where you had to make tough decisions about the architecture of a project. Where did you, or would you, draw the line on taking the "next step" in building an architecturally sound, and scalable system?

I've built a large scale web site that was rather collapsed in terms of architecture. There was a web layer with the front-end code, then business and data layers that handled the real work to be done. The various layers of logical separation co-existed on the same physical machine. A physical, or even simply logical, separation could have existed through the use of a web services layer/tier. For various reasons, it wasn't implemented that way. Whether the decision was right or wrong is simply a matter of opinion. I've been in other situations where a relatively simplistic application was over engineered, from my point of view.

What are some of the factors you consider while designing the architecture for a new project? Do you have a consistent project design you often use, are you n-tier from the start, or do you evaluate as each project comes in?

Having these experiences repeatedly, I often wonder how others in the same position justify and make these considerations. I'm sure we all will have varying opinions, but I believe understanding the thought process behind the opinions will be enlightening.

Answer 1

I use Spring - it's all built-in.

Answer 2

The correct architecture for a given problem depends entirely on the problem. Your question is too general to offer a real answer, other than to say I keep the architecture as simple as I can to account for all known and expected requirements, but no simpler.

EDIT:

For "typical" business solutions, here are some of the factors that I consider:

• UI

  • Can it be web-based? What are the user interaction requirements?
  • If a classic web interface is not sufficient, can I use a more interactive technology such as Sliverlight?
  • If it has to be thick-client (yes, there are still scenarios that justify that), how serious are the deployment challenges? Small user base, large user base? Do I need to include automatic updating? Does it need to be enforced?

• Business Layer

  • Do I have performance/scalability considerations that require a physically separate business layer? (My business layers are always logically separate, and easy to physically separate if need be. I sometimes use CSLA to allow for that decision at deployment time when targeting Windows, but that's a heavy framework and not always appropriate).
  • How simple or complex are my business rules? Are they likely to evolve considerably over time? Is it worth incorporating a rules engine such as Drools?
  • Are there asynchronous processing requirements? Do I need a work queue system?
  • Are there external systems to interface with? What types of interfaces to they present? Web service, COM+, XML over HTTP, proprietary, DB, batch files, ...?

• Data Persistence

  • What ORM choices are available to me given any pre-existing platform choices/constraints?
  • Will I benefit from extensive use of stored procedures? Will there be a DBA to maintain stored procedures and modify them over time? If no DBA, I only use stored procedures where really necessary for performance. If there is a DBA, more extensive use of stored procedures gives the DBA flexibility to manage the physical architecture independent of the application (but as with all added complexity, that comes with a cost).

• Cross-Cutting

  • What are the security requirements? Is there an existing mechanism (Active Directory/LDAP/...) to be integrated with? Do I need to support role-based security?
  • What are the operational monitoring requirements? "Report this bug" functionality? Simple logging?

Answer 3

I find that it's generally very bad practice to assume performance bottlenecks up front. You can spend a lot of up front optimizing that makes no noticeable difference at the end.

We have some great refactoring tools these days and a lot of resources on development patterns. Because the tools have gotten so much better I do not spend nearly as much time as I used to in the architecture function. Very roughly my process is like this:

1. Gather Requirements
2. Prioritize requirements (don't spend a lot of time on gold plating features)
3. I generally start with 2 tier (UI / Data&Business logic) unless I know that the Data & Business logic tiers will be separated up front.
4. For each requirement, first make it work. No patterns here unless it's painfully obvious that it is needed. I find that the need for patterns emerges in the implementation.
5. After it works, clean up the code, identify places for patterns and implement them only if you need to
6. If performance is a requirement, do performance testing, refactor as necessary.

If you work in this manner, you'll find that you err on the side of simplicity. Patterns, 3rd party tools, etc can be totally awesome at solving specific problems, but I like to keep in mind that every time I add something like that, it raises the bar of understanding required to maintain the application later. So I start simple, and add complexity only when it specifically gains me something.

I actually get a pretty bad taste in my mouth when dealing with other architects who even for a small, simple application will reach for a Dependency Injection framework, Nhibernate, NUnit, roll their own logging library, write 3x as many unit tests as they have lines of code, etc. All of these tools have specific instances where the ROI (Return on Investment, "bang for your buck") is very good, and other cases where it isn't. A good architect provides as much value as they can at the lowest time/cost possible.

Answer 4

My observation is that really good architects take the time to understand deeply the known requirements and use considerable judgement in understanding where future flexibility is provided for.

They also understand the difference between logical and physical separation of tiers.

All too often I see one of two patterns:

• This worked on the last project so we use it here ... even though the requirements are different.
• That didn't work before so we won't use it ... even though the reason it didn't work was that the implmentation was done badly

(If the only architecture problems you need address are how many tiers to have in your solution then you are indeed lucky :-)

Answer 5

Well, let me be the one to tell you - simply do it. Concentrate on whatever requirements you have now but do not try to address all possible future features, imaginary requirement changes and the various courses of development.

There is a great article written by Joel: Don't Let Architecture Astronauts Scare You.

Analyze whatever requirements you have, whatever features your software needs, look at your previous experience with similar projects and go for it.

A great architecture is never born right out of the first brain storming session. You start with one approach, adjust your course as the weather changes, have code review sessions that will yield ideas to improve the architecture, refactor some bad code pieces into good and reusable components, then finally your garage will be transforming into a castle.

Follow the KISS principle and avoid premature optimization.

Do you have a consistent project design you often use?

Of course. An individual or a team develops his own style, the techniques to solve typical problems, reusable components which altogether will form your tool set. Why would your throw them away each time your start a new project?

are you n-tier from the start?

I try to be. It serves the goals of consistency, clean structure and the separation of concerns.

or do you evaluate as each project comes in

That as well. There may be a different way to address the problem and solve it in the most efficient manner.