Managing execution priorities and request expiry time in your web application

Question

Some installations that run our applications can be under hefty stress on a busy day. Our clients ask us is there is a way to manage priorities in our application. For example, in a typical internet banking application, banks are interested in having the form “Transfer money” responsive, while the “Statement” page is a lot less critical. Not being able to transfer money is a direct loss for the bank, while not being able to produce a statement or something similar can be fixed with an apology. AFAIK, neither can you manage different request or session timeouts in a typical web application, it is one value for the whole of your web app.

Managing message priority and expiry time is a typical feature in many middleware platforms. Something like this can be useful for a web front end as well. Do any of web servers (either java or .net) or web frameworks provide these features? How would you go about implementing it if you’d have to go for roll-your-own?

Answer 1

This is a pretty broad topic (which might also be appropriate for ServerFault), and are a number of options to consider both within your application and in the platform and network environment under which it runs. There are so many variables (e.g., CPU, memory, bandwidth and database access) to consider, it is nearly impossible to come up with a definitive answer based on your question. But, here are some examples of strategies you might employ, both of which assume your application is stateless, and you can split its functionality into multiple parts either physically or virtually (isolating transactions from statements in your situation). If these assumptions do not hold true, please clarify the question.

• You can achieve this behavior within IIS. You need to split the application into multiple sites (or do so virtually by creating multiple copies of the application). A domain/subdomain model is ideal. Your transactions might occur on www.mysite.com, while statements are generated on docs.mysite.com. You can then limit connections and throttle bandwidth on the lower priority application. If you are using ASP.NET, you can also restrict the lower priority app to specific CPU cores while leaving the other unrestricted.

• Since you are dealing with a high volume application, you are likely using a load balancer to spread traffic across multiple web servers. If your application is truly stateless (that is, no use of nasty session variables, etc.), you can likely take advantage of your load balancer to perform the traffic prioritization for you in one or more ways. For example, let's say you have N front end web servers (where N > 2). You could run two separate pools in the load balancer to spread the load for transactions over 2/3 of them and statements over the other 1/3 with failover configured between them. (Depending on your load balancer, you may be able to do this based on path or HTTP header, as examples, even without splitting up your app.) This gives you both fault tolerance and prioritization. There are other choices depending on the load balancer employed, but hopefully this is enough to spark your interest in moving the problem to infrastructure and not just stay within the walls of the application server.

Answer 2

Coldfusion allows setting of request timeout on a per-request basis.

Apache allows setting expires header based on arbitrary conditions (like the URL visited).

You can run more than one webserver and set different CPU priority for each, or a load balanced cluster with more nodes for a given application than general use nodes.

To roll your own would be simple provided you know how to set thread priorities and some basic HTTP header stuff (like Expires).

You'll get a better answer if you're less vague. You ask if this can be done in .net or java yet you talk about your application like it is already written, so which is it written in, .net or java? You can hardly assume a solution for one will be applicable to the other or are we to assume you're going to rewrite your whole application? What web server are you using, that's equally relevant to the question?

Have you considered just buying more resources? For the cost of implementing and maintaining your priority schemes you could probably buy enough extra resources to make it redundant. Unless you're getting slahdotted every second day you should always have enough resources to handle at least twice your peak load or you're simply asking for trouble.