Updating a module level shared dictionary

Question

Hi,

A module level dictionary 'd' and is accessed by different threads/requests in a django web application. I need to update 'd' every minute with a new data and the process takes about 5 seconds.

What could be best solution where I want the users to get either the old value or the new value of d and nothing in between.

I can think of a solution where a temp dictionary is constructed with a new data and assigned to 'd' but not sure how this works!

Appreciate your ideas.

Thanks

Answer 1

Probably best -- at module level:

import threading
dlock = threading.Lock()
d = {}

and every access to d (not just modifications!) is within a with block:

with dlock:
    found = k in d

and the like (if you're on Python 2.5, you'll also need to have from __future__ import with_statement at the top of your module).

This protects d with the lock, so changes are serialized. The reason the guard is also needed around non-modifying access to d is that you might otherwise get problems even in "read-like" operations (if k in d:, d.get(k), etc) if the dict gets "changed from right under the operation smack in the middle of it".

Alternative architectures can be based on wrapping the dictionary (either to protect all of its needed methods with the lock, or to delegate a special purpose thread to perform all the dictionary operations and communicate with all other threads via Queue.Queue instances), but I think that in this particular case the simple, no-frills solution works for the best.