Limit the rate of requests to an API in a CGI script

views:

answers:

Limit the rate of requests to an API in a CGI script

The script that I'm writing sometimes makes requests to an API and the API requires that requests are limited to a maximum of 1 per second.

What is the most straight forward way of limiting my requests to the API to 1 every second?

Would it involve storing the current time in a file each time a request is made?

+1 A:

You could use a separate thread for the CGI calls and a queuing mechanism that loops with a call to sleep on each iteration.

From 15.3. time

time.sleep(secs) Suspend execution for the given number of seconds. The argument may be a floating point number to indicate a more precise sleep time. The actual suspension time may be less than that requested because any caught signal will terminate the sleep() following execution of that signal’s catching routine. Also, the suspension time may be longer than requested by an arbitrary amount because of the scheduling of other activity in the system.

Metalshark 2010-07-11 12:03:35

How would this separate thread work? The problem is that the script is launched each time a request is made, how do I have something that stays between requests?

Acorn 2010-07-11 12:08:27

You need persistence. So as you say in the question either secondary storage in a file or use IPC and a separate process to make the CGI calls. Be careful with edge cases in the former: request 1 may read the time, then request 2 before request 1 has had time to write the new timestamp (you'll need a file lock for this) or if you have a summer time then it may mess up when the time changes.

Metalshark 2010-07-11 12:10:46

I'd prefer not to have a process that would have to be constantly running. If I store the current time each time a request is made, what's the most efficient way of doing this? Pickling and saving to a file, and then opening the file each time a new request is made? Oh right, just saw your edit. Yes, I was thinking there could be complications.

Acorn 2010-07-11 12:15:02

You can use any form of secondary storage (file, database, person with a chalkboard) - the efficiency will vary wildly depending on your setup. If you have easy access to a transactional database then the locking may be best served there (and you can log the results). Also if the database does triggers you can write the record for the request and let the trigger do the actual call.

Metalshark 2010-07-11 12:25:35

ansaurus

tags:

views:

answers:

Limit the rate of requests to an API in a CGI script

related questions