Techniques for writing a scalable website.

Question

I am new in the website scalability realm. Can you suggest to me some the techniques for making a website scalable to a large number of users?

Answer 1

1. Test your website under heavy load.
2. Monitor all statistics
3. Find bottleneck
4. Fix bottleneck
5. Go back to 1

good luck

Answer 2

You might want to look at this resource- highscalability.com.

Answer 3

Develop your site using solid OOP techniques. You will need your site to be modular as not all performance bottlenecks are obvious at the start. Be ready to refactor parts of your site as traffic increases. The first sentence I wrote will help you do it more easily and safely. Also, use test driven development, As refactor means new introduced bugs, and good TDD is good in catching them before they go into production.
Separate as much as possible client side code from server side code, as they will likely to be served from different servers, if your site traffic justify this.
Read articles (read the YSlow tips for instance).

GL

Answer 4

Check out this talk by Rasmus Lerdorf (creator of PHP)

Specially Page 8 and beyond.

Answer 5

If you expect your site to scale beyond the capabilities of a single server you will need to plan carefully. Design so the following will be possible:-

• Make it so your database can be on a separate server. This isn't normally too hard.
• Ensure all your static content can be moved to a CDN, as this will normally pull a lot of load off your servers.
• Be prepared to spend a lot of money on hardware. More RAM and faster disks help a LOT.
• It gets a lot harder when you need to split either the database or the php from a single server to multiple servers, so optimise everything, from your code, your database schema, your server config and anything else you can think of to put this final step off for as long as possible.

Other than that, all you can do is stress test your site, figure out where the bottlenecks are and try and design them away.

Answer 6

Very similar: How Is PHP Done the Right Way?

Scalability is no small subject and certainly more material than can be reasonably covered in a single question.

For instance, with some kinds of applications, joins (in SQL) don't scale, which brings up all sorts of caching and sharding strategies.

Beanstalk is another scalability and performance tool in high-performance PHP sites. As is memcache (different kind).

Answer 7

In order of importance:

1. If you run PHP, use an opcode cache like APC. (This is important enough to be built-in in the next generation of PHP.)

2. Use YSlow or Google Page Speed to identify bottlenecks. (This will reveal structural problems with your website that affect both client and server performance.)

3. Ensure that your web server sends a proper Expires header for static content (images, Javascript, CSS), such that the browser can cache it properly. (YSlow will warn you about this, too.)

4. Use an HTTP accelerator, such as Varnish. (This picture says it all – and they already had an HTTP accelerator in place.)

Answer 8

A number of people have mentioned tools for identifying bottlenecks, and that is of course necessary. You can't spend productive time speeding something up without knowing where it's slow. But the other thing you need to know is where your target scalability lies. Is it value for money to spend a couple of months making your site scale to the same number of users as Twitter if it's going to be used by three people in HR? Do you have a known rate of transactions, or response latency, or number of users, in the requirements of the product? If so, target those numbers with your optimisation strategy. If not, find those out before chasing the performance rat down the hole.

Answer 9

The biggest problem for scalability is usually shared resources like DBMS's. The problem arises because DBMS's usually have no way to relax consistency guarantees.

If you want to increase scalability when you use something like MySQL you have to change your schema design to relax consistency.

For instance, you can separate your database schema to have your normalized data model for writes, and a replicated read only denormalized part for the 90% of read operations. The read only data can be spread over several servers.

Another way to increase scalability of a database is to partition the data, e.g. separate the data into a database for every department and aggregate them either in the ORM or in the DBMS.

Answer 10

In addition to the other suggestions, look into splitting your sites into tiers, as in multitier architecture. If done right, you can then use one server per tier.