views:

536

answers:

2

My Django site recently started throwing errors from my caching code and I can't figure out why...

I call:

from django.core.cache import cache
cache.set('blogentry', some_value)

And the error thrown by Django is:

TransactionManagementError: This code isn't under transaction management

But looking at the PostgreSQL database logs, it seems to stem from this error:

STATEMENT:  INSERT INTO cache_table (cache_key, value, expires) VALUES (E'blogentry', E'pickled_version_of_some_value', E'2009-07-27 11:10:26')
ERROR:  duplicate key value violates unique constraint "cache_table_pkey"

For the life of me I can't figure out why Django is trying to do an INSERT instead of an UPDATE. Any thoughts?

+1  A: 

The code in core/cache/backend/db.py reads in part:

cursor.execute("SELECT cache_key, expires FROM %s WHERE cache_key = %%s" % self._table, [key])
try:
    result = cursor.fetchone()
    if result and (mode == 'set' or
            (mode == 'add' and result[1] < now)):
        cursor.execute("UPDATE %s SET value = %%s, expires = %%s WHERE cache_key = %%s" % self._table, [encoded, str(exp), key])
    else:
        cursor.execute("INSERT INTO %s (cache_key, value, expires) VALUES (%%s, %%s, %%s)" % self._table, [key, encoded, str(exp)])

So I'd say that you are doing the INSERT INTO instead of the UPDATE because result evaluates to false. For some reason, cursor.fetchone() returns 0 rows when there is actually one there.

if you can't break in a debugger here, I'd put trace statements into the source to confirm that this is actually happening.

hughdbrown
yeah, I'd read the source over and over... as Glenn pointed out, the real culprit here is that the db backend has potential concurrency problems where a second INSERT statement can occur after the select returns 0, causing the original cache.set() to fail which then triggers a non-functional transaction.rollback() call.
Gabriel Hurley
+2  A: 

That's a typical race. It checks if the key you inserted exists; if it doesn't, it does an insert, but someone else can insert the key between the count and the insert. Transactions don't prevent this.

The code appears to expect this and to try to deal with it, but when I looked at the code to handle this case I could see immediately that it was broken. Reported here: http://code.djangoproject.com/ticket/11569

I'd strongly recommend sticking to the memcache backend.

Glenn Maynard
I'd read through the source for the db cache backend and was thinking the same thing about the ticket you opened. Glad it wasn't just me feeling like I was missing something here. We're gonna set up memcached instead. Reading the source for the mecached backend is so much more reassuring. Thanks!
Gabriel Hurley