Fastest way to track API access

Question

I'm creating an API for my Rails app, and I want to track how many times a user calls a particular API method, and cap them say at like 1,000 requests per day. I'm expecting very high request volumes across multiple users.

Do you have a suggestion as to how I can keep track of something like that per user? I want to avoid having to write to the database repeatedly and deal with locks.

I'm okay doing a delayed write (API limit don't have to be super exact), but is there a standard way of doing this?

Answer 1

It really depends on the # of servers, the dataset, the # of users, etc.

One approach would be to maintain a quota datastructure in memory on the server and update it per invocation. If you have multiple servers you could maintain a memcache of the quota. Obviously, a memory-based implementation wouldn't survive a reboot or restart, so some sort of serializiation would be required to support that.

If quota accuracy is critical it's probably best to just do it in the DB. You could do it in a file, but then you face the same issues you're trying to avoid /w the database.

EDIT: You could also do a mixed approach -- maintain a memory-based cache of user|api|invocation counts and periodically write them to the database.

A bit more info on the requirements would help pare down the options..

Answer 2

You could try Apigee. It looks like it's "free up to 10,000 messages per hour".

Disclaimer: I have never used Apigee.

Answer 3

Here's a way of doing it using the rails cache

call_count_key = "api_calls_#{params[:api_key]}_#{Time.now.strftime('%Y-%m-%d-%Hh')}"
call_count = Rails.cache.read(call_count_key) || 0
call_count += 1
Rails.cache.write call_count_key, call_count

# here is our limit
raise "too many calls" if call_count > 100

This isn't a perfect solution as it doesn't handle concurrency properly and if you're using the in memory cache (rails' default) then this will be a per process counter

Answer 4

If you're ok with a hosted solution, take a look at my company, WebServius ( http://www.webservius.com ) that does API management (issuing keys, enforcing quotas, etc). We also have a billing support, so that you will be able to set per-call prices, etc.