python threads - how do "condition.wait" and "condition.notifyAll" work

Question

Hi,

I have the following "consumer" code:

    ....

    while 1:

        time.sleep(self.sleeptime)

        cond.acquire() #acquire the lock
        print currentThread(), "lock acquired"

        while itemq.isEmpty():
            cond.wait()

        itemq.consume()
        print currentThread(),"Consumed One Item"
        cond.release()

And the following producer's code:

....     
while 1 :


           cond.acquire() #acquire the lock
           print currentThread(), "lock acquired"
           print currentThread(),"Produced One Item"
           itemq.produce()
           cond.notifyAll()
           cond.release()

           time.sleep(self.sleeptime)

I'm running the program with 1 producer and 2 consumers. I don't know what result to expect. The producer calls "notifyAll()", so I expect both consumers to wake up from their "wait". I see that indeed both consumer acquire the lock, but only the first one who acquired the lock actually get to consume the item. Could somebody please tell me how the "wait" command works? If both threads get the "notifyAll", how is it that only one gets to consume?

Thanks, Li

Answer 1

I think [the docs][1] are very clear:

The wait() method releases the lock, and then blocks until it is awakened by a notify() or notifyAll() call for the same condition variable in another thread. Once awakened, it re-acquires the lock and returns. It is also possible to specify a timeout.

and:

Note: the notify() and notifyAll() methods don’t release the lock; this means that the thread or threads awakened will not return from their wait() call immediately, but only when the thread that called notify() or notifyAll() finally relinquishes ownership of the lock.ownership of the lock.

Only one thread can have the lock at any time, of course: that's the core purpose of having a lock in the first place, after all!

So, IOW, notifyAll puts all waiting threads in ready-to-run state, and intrinsically all waiting to acquire the lock again so they can proceed: once the notifier releases the lock, one of the threads waiting to acquire that lock does acquire it (the others, if any, keep waiting for the lock to be released again, of course, so that only one thread has the lock at any given time).

[1]: http://docs.python.org/library/threading.html#condition-objectspossible to specify a timeout.ownership of the lock.

Answer 2

The key is in the loop around the wait:

while itemq.isEmpty():
        cond.wait()

cond.wait() is implemented something like this (example only):

def wait():
    cond.release()
    wait for notify
    cond.aquire()

So only one consumer exits the 'wait' function at a time thanks to the lock. The first consumer to exit the wait function detects that itemq.isEmpty() == false and goes on to consume the item. They then re-enter the wait function and release the lock.

The second consumer exits, detects that itemq.isEmpty() == true again, and re-enters wait() right away.