How to make Dynamic Website High Scalable?

Question

Hey
How can we make a dynamic website or a website which is developed in PHP and backend as MySQL a Social Networking Site High Scalable?

Answer 1

Before you make it highly scalable, make sure it's highly available. Redundancy in everything is key. Multiple servers, multiple sites, multiple everything!

Answer 2

Memcached

Answer 3

Worry about scaling when you need to worry about scaling, not before. At that point you should know the answer or be able to hire someone who does.

Let me attempt to clarrify this answer a little.

Scaling is a fairly convaluted term in modern development because very often "best practices" such as using design patterns, frameworks, database abstraction, and normalization are the root cause of bottlenecks which limit scalability in extremely demanded websites.

Throwing hardware at software will basically allow you to scale anything to a point.

At that point (where you need to throw more hardware at the problem) you most likely have enough of a user base that you either are making money off of your website/service or in the case of an open source product you'll likely have a following. These resources will provide you access to higher level computer scientists/engineers which can help with scaling.

Scaling at the level of reducing stress on hardware becomes very anti-"best practices". You begin writing code to do very specific tasks quickly. Take twitter for example, they've basically rewritten twitter to no longer use ruby on rails, but instead use scala and designed twitters already successful features around performance alone. Facebook on the other hand has written custom PHP packages in order to compile PHP.

Answer 4

Look at the infrastructure of sites like facebook, twitter and youtube on High Scalability. They give you a really nice overview of tools that are out there (most of them are open source and free).

You should probably look into:

• Reverse Proxying/Load Balancing (squid or varnish)
• Data caching (memcache and memcached)
• Possible backend in c++
• NoSQL (Cassandra, CouchDB, MemcacheDB)

I wrote a post on the topic a couple weeks ago if you're interested, check it out here.

Answer 5

Following good design and programming practices such as low coupling. Try finding the best solution for your site (design patterns, OO best practices) with scalability in mind.

I don't think there's any silver bullet here, it depends on each case.

Redundancy is quite important too as stillstanding says.

Still, first focus on making the best design possible, and when the time comes, worry about scalability.

Answer 6

That is like asking how to build a skyscraper, the answer is complicated and depends on a lot of things.

But these questions and suggestions should help a lot.

Diagram out your system design. Create Logical Partitions for Services, that can be split over multiple services. Create multiple virtual host names for services that can be logically split, like static image hosting. Determine which services need to be persistent over multiple requests and develop a session location and caching system.

Learn about MemCached!

Put SQL Caching in front of your sql requests.

Start thinking about "the cloud" - consider creating "disconnected" virtual images that can talk to each other, and exchange all state information in a structured way, that is server agnostic. When you get this right, you can add more cloud servicing transparently, if you get the design right.

Answer 7

As has been mentioned, scalability of an application depends on many factors. Some options may be application dependent, but proper planning is always important from the start.

There are two types of scaling, vertical and horizontal. Vertical scaling is the old "throw more hardware at it" approach, meaning upgrade the server processor and/or memory. Horizontal scaling is the proper way to scale, by adding redundancy and spanning resources across multiple servers. Usually when people talk about scaling, they mean horizontal scaling.

A typical Web architecture for scaling requires at least one load balancer, two or more Web servers, and one or more database servers. Each layer scales independently, and has to be handled differently. For example, you might add more load balancers to make the site more highly available (so if one balancer fails, the others are unaffected); this layer shouldn't require changes to your application, but will require other changes to your DNS and Web server software. Conversely, adding database servers may add to high availability, increase response times, or both. Changes to the database layer will probably require changes to the application so it can split reads and writes across multiple servers.

Of course, there are many other tools which may be used for scaling and/or high availability. Examples include application level caching (memcached), HTTP accelerators (basically HTTP level caching), and alternative data storage methods.

The best thing to do is keep these things in mind when planning and building the foundation of the application. Will it need to read and write from different database servers eventually? How are sessions stored, on the local filesystem (not scalable) or in a central cache (scalable)? Can these data objects be cached and if so, for how long before they are too stale? If you pay attention to these main things, most of the time your application will be able to scale without much additional work to the code base.