How to test a web API throttle limit in Python

Question

I would like to test a web API throttle limit of a given site using Python.

This API throttle limit allows X requests MAX over Y seconds per IP.

I would like to be able to test the reliability of this throttle limit, in particular on border cases (X-1 requests, X+1 requests)

Could you suggest a good way to do it?

Answer 1

I would write a script to do the following:

1. Make a burst of X requests, timing each request (I would use time.time()). There should be no evidence of throttling in the timing results. You may need to parallelize to hit the limit if latency is significant.
2. Make another request and time it. It should be throttled, and that should be evident in the time taken.

Update: here's sample code for HTTP requests:

import time
import urllib2

URL = 'http://twitter.com'

def request_time():
    start_time = time.time()
    urllib2.urlopen(URL).read()
    end_time = time.time()
    return end_time - start_time

def throttling_test(n):
    """Test if processing more than n requests is throttled."""
    experiment_start = time.time()
    for i in range(n):
        t = request_time()
        print 'Request #%d took %.5f ms' % (i+1, t * 1000.0)
    print '--- Throttling limit crossed ---'

    t = request_time()
    print 'Request #%d took %.5f ms' % (n+1, t * 1000.0)


throttling_test(3)