Mongrel cluster and multi-applications problem

Question

Hi, we're running in this issue. We're using a web service (using soap4r) to run some kind of searches and the problem appears when the webservice server is down and our aplication is trying to connect to it. At that point the application is unreachable, and all the customers are blocked.

What can we do to avoid that? Is possibile to block the routing to a mongrel that it is blocked? (I suppose that the apache's proxy uses a round-robin algohritm)

Thanks Roberto

Answer 1

Are you using mod_proxy to communicate between Apache your Mongrel servers?

Answer 2

Yes, I do.

Answer 3

This sounds to me like it may be more of a mod_proxy problem than one with Mongrel or Apache - are you setting Proxy Timeouts as described here?

Answer 4

Use mod_rewrite to reurn an error page that is more useful to the web service clients than Apache's proxy timeout errors. For example this Apache configuration will render maintence.html if it exists. So all you have to do is drop the file in and the server is automatically in maintence mode.

# Check for maintenance file and redirect all requests
RewriteCond %{DOCUMENT_ROOT}/system/maintenance.html -f
RewriteCond %{SCRIPT_FILENAME} !maintenance.html
RewriteRule ^.*$ /system/maintenance.html [L]

You could make it an XML file set up with the appropriate conent types that makes the static file look like a web service response that is essentially "please wait 10 minutes and try again", possibly with a configurable wait time.

Capistrano's web:disable task may be helpful to you, as it takes an argument for how long the site will be unavailable and puts that in the HTML when it generates the maintence.html file.

Answer 5

You need to put a timeout on your Soap4R requests. Set the connect_timeout attribute to something rather small. This should cause an exception to be handled in your code and not just leave the requests hanging.

You should also make sure your timeouts are set in your mod_proxy_balancer config.