views:

19

answers:

1
import threading
import weakref
_mainlock = threading.RLock()
_job_locks = weakref.WeakValueDictionary()
def do_thing(job_id):
    _mainlock.acquire() #Dictionary modification lock acquire
    _job_locks.setdefault(job_id, threading.RLock()) #Possibly modifies the dictionary
    _mainlock.release()
    _job_locks[job_id].acquire()
    try:
        one_time_init(job_id)
    finally:
        _job_locks[job_id].release()
    #On function return, the weakref.WeakValueDictionary should cause the key to evaporate

Assuming do_thing() is called many times on many threads with id numbers that may or may not be the same (say, 4 times with ID 3 and one time each with different IDs), is this thread safe? Will one_time_init() ever run more than once for a particular job ID at a time? (PS: one_time_init saves it's state that is has been run once for each ID, so calling it is a no-op if it has already run to completion)


Updated code (thanks THC4k):

import threading
import weakref
_mainlock = threading.RLock()
_job_locks = weakref.WeakValueDictionary()

def do_thing(job_id):
    with _mainlock:
        jl = _job_locks.setdefault(job_id, threading.RLock())
    with jl:
        one_time_init(job_id)
+1  A: 

Seems very safe. Why do you even need the _job_locks if one_time_init checks again if it was run? You could add the lock there. Why RLock instead of Lock (the function seems to never re-enter)?

Anyways, the with statement looks way nicer:

import threading
import weakref
_mainlock = threading.RLock()
_job_locks = weakref.WeakValueDictionary()

def do_thing(job_id):
    with _mainlock:
        _job_locks.setdefault(job_id, threading.RLock())
    with _job_locks[job_id]:
        one_time_init(job_id)
THC4k
Yeah, probably my original design blinding me to simplicity. Apparently a regular Lock can't be weakref'd. Also, I realize that there is a small bug in our examples - threading.RLock() will be dropped from the dictionary because we're not storing it as the return value from setdefault. I'll update the main post to look more like yours, and accept your answer.
CB