How do you avoid this race condition in Python / Django / MySQL?

Question

I have a model MyModel that has a field expiration_datetime.

Every time a user retrieves an instance of MyModel I need to first check if it has expired or not. If it has expired, than I need to increment some counter, update others, and then extend the expiration_datetime to some time in the future.

So the view would do something like:

if object.expiration_datetime < datetime.datetime.now(): 
    object.counter = F('counter') + 1 
    object.expiration_datetime = F('expiration_datetime') + datetime.timedelta(days=1) 
    object.save() 

There's a race condition in the code above. Say thread 1 checks and finds that the current instance has expired, it proceeds to increment the counter and extend the expiration datetime. But before it could do so, thread 2 is scheduled and does the same. By the time thread 1 finally finishes, counter has been incremented twice and expration_datetime has been extended twice.

This looks like it should be a pretty common issue. What is the most efficient way to handle it? Ideally I'd like to be able to handle it within Django in a database-portable way.

Answer 1

This may be a good use case for optimistic locking. There's a few ways to do it:

• You can have a version number, and you run your UPDATE queries so that it always includes the version number in the WHERE clause, and then check if any rows were changed.
• Include every value of the record (before the changes you made) in the WHERE clause, that way you can be sure that the record you are saving is exactly the same as when you read it.

How to implement optimistic locking in Django? Check out this question: Django: How can I protect against concurrent modification of data base entries.

Answer 2

Use a database transaction. They're designed to handle cases exactly like this.

If you're using MySQL, be aware that only InnoDB tables support ACID transactions, so make sure your tables use the InnoDB engine.