What the best/correct way to setup a load balanced environment for a web application?

Question

I develop a web application that needs to be load balanced across 4 webservers. My company is purchasing the load balancer hardware (Cisco). There is confusion on how to tell the load balancer that a webserver should be in the rotation.

The IT Hardware support team wants me to create a one page web application that will sit next to the main web application. I was told that this small one page app should call the database and do an INSERT, UPDATE, and DELETE on a table. Then the app should return a status code, which in turn the load balancer will be able to read and determine whether to keep this webserver in the rotation. This app will have to reside on all of the load balanced servers. The one page app would then be polled every 5 seconds by the load balancer.

I think that this sounds like an absolute HACK. It sounds like unnecessary work and overhead on the database. The hardware team is trying to convince me that this is the right way to go. I completely DISAGREE! My argument is that the load balancer should not care about the database connection. We are only trying to distribute the load across the 4 webservers. One thing to note is that the Main Web Application is useless without its database connection.

Does the load balancer itself have a way to determine if a server should be in the rotation? What is the correct way to do this?

Answer 1

I always put a real simple html page on the webserver. If the HTTP status from status.html comes back 200, then keep the server in the rotation, if not, suspend the web server.

What if your DB goes down hard? while you are restoring, you should be able to place a status message that says "sorry, back soon". You don't want to have to reconfigure your load balancer as well. Is there no part of your application that can run without the DB?

Answer 2

Read Scalable Internet Architectures.

The balancer can only see the web servers. If the web server appears up, but can't talk to the database (because, say, Apache is running but your app is broken), then you'll have a problem with the non-responsive server getting requests when the app is broken.

We have a selftest app that returns some status. Currently used for debugging. In the future, will be used with other HA and load-sharing solutions.

Answer 3

It is not a hack, but a common way to check the heartbeat of web servers. However, querying database in that page is controversial. For example, you may end up with no web servers in cluster in case of DB timeout.

Answer 4

I believe it is a hack. The load balancer should just use a simple html page to return to the load balancer like JWMiller said. we use this approach accross all our apps. Checking database connections, remoted services, etc. is the job of an application healthcheck and should be a separate function.