Design for scalability/performance/security etc..

Question

Most of the design articles and books are focused on getting loosely coupled and highly cohesive software..

A general guideline not to design for performance but just write code that is efficient. The performance can later be fixed at hot-spots. I am talking in term of rich-domain applications..

What about scalability and security? When do you start considering theses criterion? I am not talking in general about SQL injection..

For ex: If I have to start serving millions of users from say few hundred... When I have to put my internally used website on internet? etc etc..

Answer 1

Security should always be a consideration from the beginning. Tacking on security after the fact is a sure way to ensure that your application will never have the needed security. Scalability is something that can be tacked on later without too much disruption to the code base, assuming you at least thought about scaling when you were coding (no implementation needed).

Answer 2

Security I think is something that needs to be considered from the beginning as it can be harder to re-work security related items.

I believe that to a certain point scalability is like performance where tweaks can be made later if you need to as long as you are focused on writing efficient code overall. But that isn't a hard and fast rule. I would say that depending on your language of choice that some scalability items should be considered from the beginning, such as the use of Session in ASP.NET...

Answer 3

Could you give any example..

How would a domain model or infrastructure classes (DB Access layer etc) be affect with and without security considerations?

Answer 4

1. Security should be addressed from the very beginning. It is much easier to optimize for performance than it is to try to patch a security hole.
2. Scalability can be factored in the design process on what servers, db's, software to use.
3. Efficient code tends to be high perforance anyway so that is not as large of a issue.

Answer 5

Scalability is something that can not be achieved after developing entire application. If you have to meet the criteria of millions of hits then you need to think well before you write a single line of code.

There is a concept of scaling vertically and scaling horizontally. Scaling vertically can be achieved by throwing more memory and more processing power on a problem. But it will eventually run out. You should always plan for scaling horizontally.

Scale in a way that you can divide your business by High-usage, medium-usage and low-usage. You can then put high-usage workflows in different app/web servers. Your high usage workflow may be hitting different databases. Can you break the database load and spread the load on different boxes? If yes, then spread it at database level also. Following is a wonderful description of eBay architecture describing how they managed to scale horizontally:

http://www.addsimplicity.com/downloads/eBaySDForum2006-11-29.pdf

Answer 6

Performance and Scalability are never a problem ... until they are a problem. Your general approach of writing good quality code and then dealing with hotspots is a good plan as long as you continue to test the performance and measure it throughout the development. When working with large projects, it is always about the features, features, featues, but without regular measurements of performance you do not know if the addition of a specific feature is causing problems. Going back to find the problem 3 years later is a much more daunting task. So, I would say don't worry about performance, but start measuring it as part of your daily/weekly/iteration builds. It will provide you with great insight and confidence in your code.

Answer 7

Security from the beginning, that's tough to change later.

As for performance and scalability, there's an interview with the creators of stackoverflow on Hanselminutes: http://www.hanselminutes.com/default.aspx?showID=152

I like the pragmatic approach they took with this site, sometimes even made me cringe. For example, they have DB and web server on the same machine. Oh, and the dev server, too. But it works. And setting up another separate DB-server is easy enough once there's too much traffic for one machine.

Answer 8

Security concern is a must for all but trivial internal projects. As for scalability/performance/availability, it really depends on the service you're trying to provide. For most sites (pretty much all sites except true (not meta) web scale search engines), performance and scalability are usually not a huge issue and can be simply solved by sticking a few load-balancers in front of webapp servers with caching (e.g., using memcached etc) and adhoc data sharding (partition tables by user id etc.). The only headache usually comes from requirement of high data availability as most stock solutions for DB replication and failover don't work very well.

If you want to build a search engine for the whole web, you'd have to think about performance, scalability from day one, or you'd be just wasting your time and resources.