Scaling LAMP

Question

I have a client with a LAMP website serving mostly video. He is currently on one server with all components. He is having some scaling problems. What are some of the techniques that can be used to help.

I used separating out the DB to another server with a GB Ethernet between it and the webserver. Maybe adding more web servers with some load balancing and additional MySQL servers with replication?

Would like some medium, large, super large examples on how to scale if possible.

The video is actually coming as jpg images. Similiar to this website:

http://www.webcams.travel/webcam/1170170095

And to add some details. I figure the max visitors per hour would be 1000, and I think would be lucky to have that. Maybe closer to 1000 a day.

Answer 1

You need to work out exactly what the problem is - there is no general solution.

You need to build a performance test environment with real hardware and something which can simulate lots of clients. This may be quite expensive.

If the problem is the database (which it may be) then the solution is often to split it into several databases each holding part of the data.

But I wouldn't imagine that any sane person would store video on MySQL - if he is doing that, then a refactoring may be in order.

Answer 2

First off, I agree that you must know what the bottleneck is before attempting to scale. Here some run of the mill suggestions to get you started:

1. Get video out of the database. I don't see how this would make sense in any situation.

2. Add one more server to only server the video content and leave the HTML/application to the original one. Not only does this reduce load, but it creates a performance improvement on the client side by overcoming any HTTP connection limits.

3. Cache - This may mean Memcahce or just pre-building HTML output instead of dynamically processing it for every request.

4. If you are fortunate enough to reach 'super large' you might think about a CDN. I'm sure you will encounter many other hurdles before then though.

Best of luck.

Answer 3

Profile to see how much load various parts of his site actually inflict.

I presume most of the load is actually serving the videos - use a proxy to redirect this work to a second (,third, fourth...) server.

Answer 4

Barrett's answers are pretty much what I would do -- id the bottlenecks, look at memcached, move the DB to a separate server from web services, etc.

I would add that Amazon has a new service called CloudFront that will serve up content from geographically close servers. I haven't given it a try yet, but that may be a way to spread the load for a relatively low cost.

Also, look at the presentations from Livejournal and Facebook on how they scale their systems, they may provide some insight depending on how you have your applicaiton(s) structured.

Answer 5

The advice about CloudFront and MemCache etc. is all good, assuming those address the root of your performance issues.

On the database side: Profile, profile, profile. Moving the DB to a separate server was (probably) a good step, but if you haven't profiled the queries being run on this DB instance, you don't know what you need to improve.

Start by doing an EXPLAIN on the most common queries, and check whether you're doing unnecessary sequential scans on large or growing tables. Index as necessary.

Are any of the queries passing back columns that aren't used?

Are you, as other commenters were worried, serving your graphic/video content from the database? Files should be kept in the filesystem, and served from there.

Are all your tables MyISAM? Frequently-updated tables may show performance improvement by moving them to InnoDB, where they can benefit from row-level locking (as opposed to MyISAM's table-level locking).

These are just simple, general recommendations - without more specifics on exactly what is slow, it's tough to offer anything deeper.