How to limit rate of requests to web services in Python?

Question

I'm working on a Python library that interfaces with a web service API. Like many web services I've encountered, this one requests limiting the rate of requests. I would like to provide an optional parameter, limit, to the class instantiation that, if provided, will hold outgoing requests until the number of seconds specified passes.

I understand that the general scenario is the following: an instance of the class makes a request via a method. When it does, the method emits some signal that sets a lock variable somewhere, and begins a countdown timer for the number of seconds in limit. (In all likelihood, the lock is the countdown timer itself.) If another request is made within this time frame, it must be queued until the countdown timer reaches zero and the lock is disengaged; at this point, the oldest request on the queue is sent, and the countdown timer is reset and the lock is re-engaged.

Is this a case for threading? Is there another approach I'm not seeing?

Should the countdown timer and lock be instance variables, or should they belong to the class, such that all instances of the class hold requests?

Also, is this generally a bad idea to provide rate-limiting functionality within a library? I reason since, by default, the countdown is zero seconds, the library still allows developers to use the library and provide their own rate-limiting schemes. Given any developers using the service will need to rate-limit requests anyway, however, I figure that it would be a convenience for the library to provide a means of rate-limiting.

Regardless of placing a rate-limiting scheme in the library or not, I'll want to write an application using the library, so suggested techniques will come in handy.

Many thanks for your suggestions!

Chris

Answer 1

This works out better with a queue and a dispatcher.

You split your processing into two sides: source and dispatch. These can be separate threads (or separate processes if that's easier).

The Source side creates and enqueues requests at whatever rate makes them happy.

The Dispatch side does this.

1. Get the request start time, s.

2. Dequeues a request, process the request through the remote service.

3. Get the current time, t. Sleep for rate - (t - s) seconds.

If you want to run the Source side connected directly to the remote service, you can do that, and bypass rate limiting. This is good for internal testing with a mock version of the remote service.

The hard part about this is creating some representation for each request that you can enqueue. Since the Python Queue will handle almost anything, you don't have to do much.

If you're using multi-processing, you'll have to pickle your objects to put them into a pipe.

Answer 2

Your rate limiting scheme should be heavily influenced by the calling conventions of the underlying code (syncronous or async), as well as what scope (thread, process, machine, cluster?) this rate-limiting will operate at.

I would suggest keeping all the variables within the instance, so you can easily implement multiple periods/rates of control.

Lastly, it sounds like you want to be a middleware component. Don't try to be an application and introduce threads on your own. Just block/sleep if you are synchronous and use the async dispatching framework if you are being called by one of them.

Answer 3

Queuing may be overly complicated. A simpler solution is to give your class a variable for the time the service was last called. Whenever the service is called (!1), set waitTime to delay - Now + lastcalltime. delay should be equal to the minimum allowable time between requests. If this number is positive, sleep for that long before making the call (!2). The disadvantage/advantage of this approach is that it treats the web service requests as being synchronous. The advantage is that it is absurdly simple and easy to implement.

• (!1): Should happen right after receiving a response from the service, inside the wrapper (probably at the bottom of the wrapper).
• (!2): Should happen when the python wrapper around the web service is called, at the top of the wrapper.

S.Lott's solution is more elegant, of course.

Answer 4

If your library is designed to be synchronous, then I'd recommend leaving out the limit enforcement (although you could track rates and at least help the caller decide how to honor limits).

I use twisted to interface with pretty much everything nowadays. It makes it easy to do that type of thing by having a model that separates request submission from response handling. If you don't want your API users to have to use twisted, you'd at least be better off understanding their API for deferred execution.

For example, I have a twitter interface that pushes a rather absurd number of requests through on behalf of xmpp users. I don't rate limit, but I did have to do a bit of work to prevent all of the requests from happening at the same time.

Answer 5

SO I am assuming something simple like import time time.sleep(2) will not work for waiting 2 seconds between requests