Whose responsibility is it to throttle web requests?

Question

I am working on a class library that retrieves information from a third-party web site. The web site being accessed will stop responding if too many requests are made within a set time period (~0.5 seconds).

The public methods of my library directly relate to a resource an file on the web server. In other words, each time a method is called, an HttpWebRequest is created and sent to the server. If all goes well, an XML file is returned to the caller. However, if this is the second web request in less than 0.5s, the request will timeout.

My dilemma lies in how I should handle request throttling (if at all). Obviously, I don't want the caller sit around waiting for a response -- especially if I'm completely certain that their request will timeout.

Would it make more sense for my library to queue and throttle the webrequests I create, or should my library simply throw an exception if the a client does not wait long enough between API calls?

Answer 1

That's the Web server's responsibility, imo. Because the critical load depends on hardware, network bandwidth, etc a lot of things that are outside of your application's control, it should not concern itself with trying the deal with it. IIS can throttle traffic based on various configuration options.

Answer 2

I'd say both - you're dealing with two independent systems and both should take measures to defend themselves from excessive load. The web server should refuse incoming connections, and the client library should take steps to reduce the requests it makes to a slow or unresponsive external service. A common pattern for dealing with this on the client is 'circuit breaker' which wraps calls to an external service, and fails fast for a certain period after failure.

Answer 3

What kind of client is it? Is this an interactive client, for eg: GUI based app?

In that case, you can equate that to a webbrowser scenario, and let the timeout surface to the caller. Also, if you know for sure that this webserver is throttling requests, you can tell the client that he has to wait for a given time period before retrying. In that way, the client will not keep on re-issuing requests, and will know when the first timeout occurs that it is futile to issue requests too fast.

Answer 4

The concept of a library is to give its client code as little to worry about as possible. Therefore I would make it the libraries job to queue requests and return results in a timely manner. In an ideal world you would use a callback or delegate model so that the client code can operate in asynchronously, not blocking the UI. You could also offer the option for skipping the queue, (and failing if it operates too soon) and possibly even offer priorities within the queue model.

I also believe it is the responsibility of the library author to default to being a good citizen, and for the library's default operation to be to comply to the conditions of the data provider.