Load balancing: DNS round robin in front of hardware load balancers. How to share stickiness?

Question

Hi,

DNS Round Robin (DRR) permits to do cheap load balancing (distribution is a better term). It has the pro of permitting infinite horizontal scaling. The con is that if one of the web servers goes down, some clients continue to use the broken IP for minutes (min TTL 300s) or more, even if the DNS implements fail-over.

An Hardware Load Balancer (HLB) handles such web server failures transparently but it cannot scale its bandwidth indefinitely. An hot spare is also needed.

A good solution seems to use DRR in front to a group of HLB pairs. Each HLB pair never goes down and therefore DRR never keeps clients down. Plus, when bandwidth isn't enough you can add a new HLB pair to the group.

Problem: DRR moves clients randomly between the HLB pairs and therefore (AFAIK) session stickiness cannot work.

I could just avoid to use session stickiness but it makes better use of caches therefore is something that I want to preserve.

Question: is it possible/exist an HLB implementation where an instance can share its (sessionid,webserver) mapping with other instances?

If this is possible then a client would be routed to the same web server independently by the HLB that routed the request.

Thanks in advance.

Answer 1

Modern load balancers have very high throughput capabilities (gigabit). So unless you're running a huuuuuuuuuuge site (e.g. google), adding bandwidth is not why you'll need a new pair of load balancers, especially since most large sites offload much of their bandwidth to CDNs (Content Delivery Networks) like Akamai. If you're pumping a gigabit of un-CDN-able data through your site and don't already have a global load-balancing strategy, you've got bigger problems than cache affinity. :-)

Instead of bandwidth limits, sites tend to add additional LB pairs for geo-distribution of servers at separate data centers to ensure users spread across the world can talk to a server closest to them.

For that latter scenario, load balancer companies offer geo-location solutions, which (at least until a few years ago which was when I was following this stuff) were based on custom DNS implementations which looked at client IPs and resolved to the load balancer pairs Virtual IP address which is "closest" (in network topology or performance) to the client. These days, CDNs like Akamai also offer global load balancing services (e.g. http://www.akamai.com/html/technology/products/gtm.html). Amazon's EC2 hosting also supports this kind of feature for sites hosted there (see http://aws.amazon.com/elasticloadbalancing/).

Since users tend not to move across continents in the course of a single session, you automatically get affinity (aka "stickiness") with geographic load balancing, assuming your pairs are located in separate data centers.

Keep in mind that geo-location is really hard since you also have to geo-locate your data to ensure your back-end cross-data-center network doesn't get swamped.

I suspect that F5 and other vendors also offer single-datacenter solutions which achieve the same ends, if you're really concerned about the single point of failure of network infrastructure (routers, etc.) inside your datacenter. But router and switch vendors have high-availability solutions which may be more appropriate to address that issue.

Net-net, if I were you I wouldn't worry about multiple pairs of load balancers. Get one pair and, unless you have a lot of money and engineering time to burn, partner with a hoster who's good at keeping their data center network up and running.

That said, if cache affinity is such a big deal for your app that you're thinking about shelling out big $$$ for multiple pairs of load balancers, it may be worth considering some app architecture changes (like using an external caching cluster). Solutions like memcached (for linux) are designed for this scenario. Microsoft also has one coming called "Velocity".

Anyway, hope this is useful info-- it's admittedly been a while since I've been deeply involved in this space (I was part of the team which designed an application load balancing product for a large software vendor) so you might want to double-check my assumptions above with facts you can pull off the web from F5 and other LB vendors.

Answer 2

Hi Justin,

thanks for having put things in the right perspective. I agree with you.

I did some reading and found:

• Flickr: http://highscalability.com/flickr-architecture
  4 billion queries per day --> about 50000 queries/s

• Youtube: http://highscalability.com/youtube-architecture
  100 million video views/day --> about 1200 video views/second

• PlentyOfFish: http://highscalability.com/plentyoffish-architecture
  600 pages/second
  200 Mbps used
  CDN used

• Twitter: http://highscalability.com/scaling-twitter-making-twitter-10000-percent-faster
  300 tweets/second
  600 req/s

A very top end LB like this can scale up :

• 200,000 SSL handshakes per second
• 1 million TCP connections per second
• 3.2 million HTTP requests per second
• 36 Gbps of TCP or HTTP throughput

Therefore, you are right a LB could hardly become a bottleneck.

Anyway I found this (old) article http://www.tenereillo.com/GSLBPageOfShame.htm where it is explained that geo-aware DNS could create availability issues.

Could someone comment on that article?

Thanks,

Valentino