Load balancing for a web service using soap ?

Question

We are currently looking at the problem of creating a load balancer in front of a set of servers offering the same web service (using soap over http).

Once a client request has been sent to a given server, that server should respond to all subsequent requests from that client (this is sometimes referred as stickiness if I am not mistaken).

My question is :does it exist good software for doing this, or should we do it ourselves ? I know there are lots of option for load balancers for web sites (html over http), but what about soap over http ?

Answer 1

If it's pure SOAP (by "pure" I mean stateless request-response style), it should be all fine and dandy. If, however, there is WS-* stuff added on top of SOAP, things may get wonky.

Answer 2

Before starting to use load balancers, read http://www.amazon.com/Scalable-Internet-Architectures-Theo-Schlossnagle/dp/067232699X.

What you want to do is use Wackamole to create a pool of resources. A load balancer can become a single point of failure. Wackamole prevents this failure mode by making all working servers into peers for load sharing.

Answer 3

Use the load balance solution. Rolling you own is very difficult. Most load balancers have this feature or it is an firmware upgrade.

I have done this before and the load balancer worked very well except for those client behind a proxy server. They all end up all going to the same server regardless of the balance algorithm.

Most proxy servers have a solution for this but getting the client to implement it is tough.

Answer 4

Is this mission critical? If so I'd really recommend you find a way to suck up the cost and build a redundant network with a pair of load balancers. We use F5 bigIP's. The low end models are not outrageously priced and there are other cheaper solutions. The reliability and manageability make it worth the cost (not to mention compression, ssl offloading etc).

P.S. To get more sophisticated features like SOAP method firewalling you need to spend a lot more dollars with F5. Does anyone know of any SOAP specific appliances that combine load balancing and SOAP specific features?

Answer 5

We are currently looking at the problem of creating a load balancer in front of a set of servers offering the same web service (using soap over http).

Don't do that, don't create your load balancer. Use an existing hardware (pretty expensive) or software solution. With most application servers, Apache (and the adequate plugin) can be used as load balancer.

Once a client request has been sent to a given server, that server should respond to all subsequent requests from that client (this is sometimes referred as stickiness if I am not mistaken).

Managing sticky session is common for web browsing and load balancers all know how to deal with session cookies but... cookies are not really designed for web services that are stateless by nature. However, the WS-I Basic Profile 1.0 doesn't completely disallow them as written here:

The HTTP State Management Mechanism ("Cookies") allows the creation of stateful sessions between Web browsers and servers. Being designed for hypertext browsing, Cookies do not have well-defined semantics for Web services, and, because they are external to the SOAP Envelope, are not accommodated by either SOAP 1.1 or WSDL 1.1. However, there are situations where it may be necessary to use Cookies; e.g., for load balancing between servers, or for integration with legacy systems that use Cookies. For these reasons, the Profile limits the ways in which Cookies can be used, without completely disallowing them.

R1120 An INSTANCE MAY use the HTTP state mechanism
         ("Cookies").

R1122 An INSTANCE using Cookies SHOULD conform to 
         RFC2965.

R1121 An INSTANCE SHOULD NOT require consumer support
         for Cookies in order to function correctly.

R1123 The value of the cookie MUST be considered to be
         opaque by the CONSUMER. 

The Profile recommends that cookies not be required by instances for proper operation; they should be a hint, to be used for optimization, without materially affecting the execution of the Web service. However, they may be required in legacy integration and other exceptional use cases, so requiring them does not make an instance non-conformant. While Cookies thus may have meaning to the instance, they should not be used as an out-of-bound data channel between the instance and the consumer. Therefore, interpretation of Cookies is not allowed at all by the consumer - it is required to treat them as opaque (i.e., have no meaning to the consumer).

So it should be possible to use cookies for stickiness but this won't be transparent for the web services consumer. There are maybe some other options (e.g. IP-based) but I'm not aware of all of them.

My question is: does it exist good software for doing this, or should we do it ourselves? I know there are lots of option for load balancers for web sites (html over http), but what about soap over http ?

Actually, I already answered that part: don't implement such a solution, use an existing one. If a software load balacing is an option, look at Apache and a plugin (like the apache weblogic plugin, mod_jk for Tomcat, etc). An hardware load balancer, something like an Alteon or a BIG-IP, might offer more possibilities (for the sticky part) but it will cost you really much more money too.

Answer 6

Membrane SOA Monitor (GUI) and Router(Commandline) are open source SOAP routers that support rule based filtering, loadbalancing and URL rewritting of endpoints and schemaLocations in WSDL documents. Membrane can also work as SOAP application level gateway. The Web Services loadbalancer is realized as a interceptor that is plugged into the router. The code of the balancer is only a few kilobytes big and can be modified to support cookies and sticky sessions.