Does bigtable need locking?

Question

I am using task queue for certain data updation in GAE.

My queue.xml file look like below

  <queue> 
    <name>data-processing</name> 
    <rate>20/s</rate> 
  </queue> 

My queue processing servlet decrease credit by 1 for every task. While processing it need to check for credit availability and proceed further only if credit is available.

The credit is stored in a table and it get updated when a task is completed.

I see the tasks as threads and worried about synchonization problem.

what if 2 or more tasks query/update credit table simultaniously? Do i need to create some locking mechenism? If yes then how?

Answer 1

Yes you do need synchronization. You'll typically update the credits in a read-modify-write scheme: first read the available credits, subtract one, and write the remaining credits back. If two tasks do this at the same time, one may overwrite the result of the other leading to an incorrect credit count being stored. (Unless there's an atomic instruction for this, which do exist for Memcache, but not for the Data Store I believe).

You can use transactions to solve this, see http://code.google.com/appengine/docs/java/datastore/transactions.html

Answer 2

The App Engine task queue takes care of accounting for execution rate itself. You don't need to do anything other than configure queue.xml as you already have.