views:

54

answers:

1

Google app engine task queues have configuration as (example)

  <queue>
    <name>mail-queue</name>
    <rate>5/m</rate>
    <bucket-size>10</bucket-size>
  </queue>

Here, what does the 'bucket-size' mean? I could not find a comprehensive documentation about this in google app engine documentation.

Does specifying this as 10 means that if 100 tasks are queued at an instant only 10 of those will be put in the queue and rest will be ignored?

+3  A: 

bucket-size is perfectly described here:

Limits the burstiness of the queue's processing, i.e. a higher bucket size allows bigger spikes in the queue's execution rate. For example, consider a queue with a rate of 5/s and a bucket size of 10. If that queue has been inactive for some time (allowing its "token bucket" to fill up), and 20 tasks are suddenly enqueued, it will be allowed to execute 10 tasks immediately. But in the following second, only 5 more tasks will be able to be executed because the token bucket has been depleted and is refilling at the specified rate of 5/s.

If no bucket_size is specified for a queue, the default value is 5.

For your case it means that if 100 messages are queued, only ten are directly being executed and another 5 every next minute. You won't loose any messages, but they will queue up if your bucket-size and rate is too low.

halfdan
+1 Amazing! Also, weird to know that 'java' task queue documentation does not have this info while 'python' documentation has it! I had an impression that java and python documentation would be content-wise similar. As I didnt find the thing under java I made an assumption that it would not be in python as well ! Thanks for pointing it out.
Gopi
Aah.. found it in java as well. But its not as comprehensive as python's. http://code.google.com/intl/en/appengine/docs/java/config/queue.html
Gopi