Why do pthreads’ condition variable functions require a mutex?

Question

I’m reading up on pthread.h; the condition variable related functions (like pthread_cond_wait(3)) require a mutex as an argument. Why? As far as I can tell, I’m going to be creating a mutex just to use as that argument? What is that mutex supposed to do?

Answer 1

It's just the way that condition variables were initially implemented in DEC threading, the precursor to pthreads.

The mutex was originally used to protect the condition variable itself. That's why you need it locked before you do a wait.

The wait will "atomically" unlock the mutex, allowing others access to the condition variable (for signalling). Then when the condition variable is signalled or broadcast to, one or more of the threads on the waiting list will be woken up and the mutex will be magically locked again for that thread.

You typically see the following operation with condition variables, illustrating how they work. The following example is a worker thread which is given work via a signal to a condition variable.

thread:
    initialise.
    lock mutex.
    while thread not told to stop working:
        wait on condvar using mutex.
        if work is available to be done:
            do the work.
    unlock mutex.
    clean up.
    exit thread.

The work is done within this loop provided that there is some available when the wait returns. When the thread has been flagged to stop doing work (usually by another thread setting the exit condition then kicking the condition variable to wake this thread up), the loop will exit, the mutex will be unlocked and this thread will exit.

The code above is a single-consumer model as the mutex remains locked while the work is being done. For a multi-consumer variation, you can use, as an example:

thread:
    initialise.
    lock mutex.
    while thread not told to stop working:
        wait on condvar using mutex.
        if work is available to be done:
            copy work to thread local storage.
            unlock mutex.
            do the work.
            lock mutex.
    unlock mutex.
    clean up.
    exit thread.

which allows other consumers to receive work while this one is doing work.

The condition variable relieves you of the burden of polling some condition instead allowing another thread to notify you when something needs to happen. Another thread can tell that thread that work is available as follows:

lock mutex.
flag work as available.
signal condition variable.
unlock mutex.

The vast majority of what are often erroneously called spurious wakeups was generally always because multiple threads had been signalled within their pthread_cond_wait call (broadcast), one would return with the mutex, do the work, then re-wait.

Then the second signalled thread could come out when there was no work to be done. So you had to have an extra variable indicating that work should be done (this was inherently mutex-protected with the condvar/mutex pair here - other threads needed to lock the mutex before changing it however).

It was technically possible for a thread to return from a condition wait without being kicked by another process (this is a genuine spurious wakeup) but, in all my many years working on pthreads, both in development/service of the code and as a user of them, I never once received one of these. Maybe that was just because HP had a decent implementation :-)

In any case, the same code that handled the erroneous case also handled genuine spurious wakeups as well since the work-available flag would not be set for those.

Answer 2

The mutex is supposed to be locked when you call pthread_cond_wait; when you call it it atomically both unlocks the mutex and then blocks on the condition. Once the condition is signaled it atomically locks it again and returns.

This allows the implementation of predictable scheduling if desired, in that the thread that would be doing the signalling can wait until the mutex is released to do its processing and then signal the condition.

Answer 3

Besides consistency I think it might be related to memory visibility. Seems like POSIX guarantees (memory visibility between threads) that two threads see memory equally after four events and one of them is pthread_mutex_unlock().

So this might look like this:

Step 1. Thread 1 has done some work and starts waiting on a conditional variable, unlocks a mutex. Unlocking the mutex guarantees (according to rule #2) that all other threads will see all modifications in memory that have been done by the thread 1. If you do not unlock any mutex at this point then according to this book it is not guaranteed that other threads will see changes done by the thread #1.

Step 2. Thread 2 changes some data and signals on the conditional variable. According to rule #4 the thread #1 will see all modifications done by the thread #2

Answer 4

A condition variable is quite limited if you could only signal a condition, usually you need to handle some data that's related to to condition that was signalled. Signalling/wakeup have to be done atomically in regards to achieve that without introducing race conditions, or be overly complex

pthreads can also give you , for rather technical reasons, a spurious wakeup . That means you need to check a predicate, so you can be sure the condition actually was signalled - and distinguish that from a spurious wakeup. Checking such a condition in regards to waiting for it need to be guarded - so a condition variable needs a way to atomically wait/wake up while locking/unlocking a mutex guarding that condition.

Consider a simple example where you're notified that some data are produced. Maybe another thread made some data that you want, and set a pointer to that data.

Imagine a producer thread giving some data to another consumer thread through a 'some_data' pointer.

while(1) {
    pthread_cond_wait(&cond); //imagine cond_wait did not have a mutex
    char *data = some_data;
    some_data = NULL;
    handle(data);
}

you'd naturally get a lot of race condition, what if the other thread did some_data = new_data right after you got woken up, but before you did data = some_data

You cannot really create your own mutex to guard this case either .e.g

while(1) {

    pthread_cond_wait(&cond); //imagine cond_wait did not have a mutex
    pthread_mutex_lock(&mutex);
    char *data = some_data;
    some_data = NULL;
    pthread_mutex_unlock(&mutex);
    handle(data);
}

Will not work, there's still a chance of a race condition in between waking up and grabbing the mutex. Placing the mutex before the pthread_cond_wait doesn't help you, as you will now hold the mutex while waiting - i.e. the producer will never be able to grab the mutex. (note, in this case you could create a second condition variable to signal the producer that you're done with some_data - though this will become complex, especially so if you want many producers/consumers.)

Thus you need a way to atomically release/grab the mutex when waiting/waking up from the condition. That's what pthread condition variables does, and here's what you'd do:

while(1) {
    pthread_mutex_lock(&mutex);
    while(some_data != NULL) { // predicate to acccount for spurious wakeups,would also 
                               // make it robust if there were several consumers
       pthread_cond_wait(&cond,&mutex); //atomically lock/unlock mutex
    }

    char *data = some_data;
    some_data = NULL;
    pthread_mutex_unlock(&mutex);
    handle(data);
}

(the producer would naturally need to take the same precautions, always guarding 'some_data' with the same mutex, and making sure it doesn't overwrite some_data if some_data is currently != NULL)

Answer 5

The race it's avoiding is actually quite simple. Consider a consumer/producer where wait();, unlock(); and lock(); were distinct:

Consumer:

while (1)
{
    wait();
    lock();
    consume();
    unlock();
}

Producer:

/* ... */
lock();
produce();
signal();
unlock();
/* ... */

Under this scheme, if the Consumer happens to be in between unlock(); and wait(); when the Producer executes signal();, it will miss the signal and potentially wait forever.

Under the scheme where wait(); implies an atomic unlock-and-lock, the Consumer becomes:

lock();
while (1)
{
    wait();
    consume();
}

Because signal(); is only done under the lock, and the only time the Consumer doesn't hold the lock is atomically within wait();, the race is avoided.