views:

473

answers:

6

I need a component/class that throttles execution of some method to maximum M calls in N seconds (or ms or nanos, does not matter).

In other words I need to make sure that my method is executed no more than M times in a sliding window of N seconds.

If you don't know existing class feel free to post your solutions/ideas how you would implement this.

A: 

Check out the TimerTask class. Or the ScheduledExecutor.

Gandalf
+2  A: 

Read up on the Token bucket algorithm. Basically, you have a bucket with tokens in it. Every time you execute the method, you take a token. If there are no more tokens, you block until you get one. Meanwhile, there is some external actor that replenishes the tokens at a fixed interval.

I'm not aware of a library to do this (or anything similar). You could write this logic into your code or use AspectJ to add the behavior.

Kevin
Thanks for suggestion, interesting algo. But its not exactly what I need. For example, I need to limit execution to 5 call per second. If I use Token bucket and 10 requests come in at the same time, first 5 calls would take all available tokens and execute momentarily, while remaining 5 calls will be executed at fixed interval of 1/5 s. In such situation I need remaining 5 call to be executed in single burst only after 1 second passes.
valery_la99
What if you added 5 tokens to the bucket every second (or 5 - (5-remaining) instead of 1 every 1/5 second?
Kevin
@Kevin no this still would not give me 'sliding window' effect
valery_la99
+4  A: 

I'd use a ring buffer of timestamps with a fixed size of M. Each time the method is called, you check the oldest entry, and if it's less than N seconds in the past, you execute and add another entry, otherwise you sleep for the time difference.

Michael Borgwardt
Lovely. Just what I need. Quick attempts shows ~10 lines to implement this and minimal memory footprint. Just need to think about thread safety and queuing of incoming requests.
valery_la99
That's why you use the DelayQueue from java.util.concurrent. It prevents the problem of multiple threads acting on the same entry.
erickson
For a multi threaded case, the token bucket approach may be a better choice, I think.
Michael Borgwardt
+3  A: 

In concrete terms, you should be able to implement this with a DelayQueue. Initialize the queue with M Delayed instances with their delay initially set to zero. As requests to the method come in, take a token, which causes the method to block until the throttling requirement has been met. When a token has been taken, add a new token to the queue with a delay of N.

erickson
Yes, this would do the trick. But I don't particularly like DelayQueue because it is using (via PriortyQueue) a balanced binary hash (which means lots of comparisons on `offer` and possible array growth), and its all kinda heavy for me. I guess for others this might be perfectly okay.
valery_la99
Actually, in this application, since the new element added to the heap will almost always be the maximum element in the heap (i.e., have the longest delay), usually one comparison per add is required. Also, the array will never grow if the algorithm is implemented correctly, since one element is added only after taking one element.
erickson
+1  A: 

Although it's not what you asked, ThreadPoolExecutor, which is designed to cap to M simultaneous requests instead of M requests in N seconds, could also be useful.

eed3si9n
A: 

I need to make sure that my method is executed no more than M times in a sliding window of N seconds.

I recently wrote a blog post about how to do this in .NET. You might be able to create something similar in Java.

Better Rate Limiting in .NET [Penned Objects]

Jack Leitch