In pthread, how to reliably pass signal to another thread?

Question

I'm trying to write a simple thread pool program in pthread. However, it seems that pthread_cond_signal doesn't block, which creates a problem. For example, let's say I have a "producer-consumer" program:

pthread_cond_t my_cond = PTHREAD_COND_INITIALIZER;
pthread_mutex_t my_cond_m = PTHREAD_MUTEX_INITIALIZER;

void * liberator(void * arg)
{
    // XXX make sure he is ready to be freed
    sleep(1);

    pthread_mutex_lock(&my_cond_m);
    pthread_cond_signal(&my_cond);
    pthread_mutex_unlock(&my_cond_m);

    return NULL;
}

int main()
{
    pthread_t t1;
    pthread_create(&t1, NULL, liberator, NULL);

    // XXX Don't take too long to get ready. Otherwise I'll miss 
    // the wake up call forever
    //sleep(3);

    pthread_mutex_lock(&my_cond_m);
    pthread_cond_wait(&my_cond, &my_cond_m);
    pthread_mutex_unlock(&my_cond_m);

    pthread_join(t1, NULL);

    return 0;
}

As described in the two XXX marks, if I take away the sleep calls, then main() may stall because it has missed the wake up call from liberator(). Of course, sleep isn't a very robust way to ensure that either.

In real life situation, this would be a worker thread telling the manager thread that it is ready for work, or the manager thread announcing that new work is available.

How would you do this reliably in pthread?

---

Elaboration

@Borealid's answer kind of works, but his explanation of the problem could be better. I suggest anyone looking at this question to read the discussion in the comments to understand what's going on.

In particular, I myself would amend his answer and code example like this, to make this clearer. (Since Borealid's original answer, while compiled and worked, confused me a lot)

// In main
pthread_mutex_lock(&my_cond_m);

// If the flag is not set, it means liberator has not 
// been run yet. I'll wait for him through pthread's signaling 
// mechanism

// If it _is_ set, it means liberator has been run. I'll simply 
// skip waiting since I've already synchronized. I don't need to 
// use pthread's signaling mechanism
if(!flag) pthread_cond_wait(&my_cond, &my_cond_m);

pthread_mutex_unlock(&my_cond_m);

// In liberator thread
pthread_mutex_lock(&my_cond_m);

// Signal anyone who's sleeping. If no one is sleeping yet, 
// they should check this flag which indicates I have already 
// sent the signal. This is needed because pthread's signals 
// is not like a message queue -- a sent signal is lost if 
// nobody's waiting for a condition when it's sent.
// You can think of this flag as a "persistent" signal
flag = 1;
pthread_cond_signal(&my_cond);
pthread_mutex_unlock(&my_cond_m);

Answer 1

Use a synchronization variable.

In main:

pthread_mutex_lock(&my_cond_m);
while (!flag) {
    pthread_cond_wait(&my_cond, &my_cond_m);
}
pthread_mutex_unlock(&my_cond_m);

In the thread:

pthread_mutex_lock(&my_cond_m);
flag = 1;
pthread_cond_broadcast(&my_cond);
pthread_mutex_unlock(&my_cond_m);

For a producer-consumer problem, this would be the consumer sleeping when the buffer is empty, and the producer sleeping when it is full. Remember to acquire the lock before accessing the global variable.

Answer 2

I found out the solution here. For me, the tricky bit to understand the problem is that:

1. Producers and consumers must be able to communicate both ways. Either way is not enough.
2. This two-way communication can be packed into one pthread condition.

To illustrate, the blog post mentioned above demonstrated that this is actually meaningful and desirable behavior:

pthread_mutex_lock(&cond_mutex);
pthread_cond_broadcast(&cond):
pthread_cond_wait(&cond, &cond_mutex);
pthread_mutex_unlock(&cond_mutex);

The idea is that if both the producers and consumers employ this logic, it will be safe for either of them to be sleeping first, since the each will be able to wake the other role up. Put it in another way, in a typical producer-consumer sceanrio -- if a consumer needs to sleep, it's because a producer needs to wake up, and vice versa. Packing this logic in a single pthread condition makes sense.

Of course, the above code has the unintended behavior that a worker thread will also wake up another sleeping worker thread when it actually just wants to wake the producer. This can be solved by a simple variable check as @Borealid suggested:

while(!work_available) pthread_cond_wait(&cond, &cond_mutex);

Upon a worker broadcast, all worker threads will be awaken, but one-by-one (because of the implicit mutex locking in pthread_cond_wait). Since one of the worker threads will consume the work (setting work_available back to false), when other worker threads awake and actually get to work, the work will be unavailable so the worker will sleep again.

Here's some commented code I tested, for anyone interested:

// gcc -Wall -pthread threads.c -lpthread

#include <stdio.h>
#include <pthread.h>
#include <stdlib.h>
#include <assert.h>

pthread_cond_t my_cond = PTHREAD_COND_INITIALIZER;
pthread_mutex_t my_cond_m = PTHREAD_MUTEX_INITIALIZER;

int * next_work = NULL;
int all_work_done = 0;

void * worker(void * arg)
{
    int * my_work = NULL;

    while(!all_work_done)
    {
        pthread_mutex_lock(&my_cond_m);

        if(next_work == NULL)
        {
            // Signal producer to give work
            pthread_cond_broadcast(&my_cond);

            // Wait for work to arrive
            // It is wrapped in a while loop because the condition 
            // might be triggered by another worker thread intended 
            // to wake up the producer
            while(!next_work && !all_work_done)
                pthread_cond_wait(&my_cond, &my_cond_m);
        }

        // Work has arrived, cache it locally so producer can 
        // put in next work ASAP
        my_work = next_work;
        next_work = NULL;
        pthread_mutex_unlock(&my_cond_m);

        if(my_work)
        {
            printf("Worker %d consuming work: %d\n", (int)(pthread_self() % 100), *my_work);
            free(my_work);
        }
    }

    return NULL;
}

int * create_work()
{
    int * ret = (int *)malloc(sizeof(int));
    assert(ret);
    *ret = rand() % 100;
    return ret;
}

void * producer(void * arg)
{
    int i;

    for(i = 0; i < 10; i++)
    {
        pthread_mutex_lock(&my_cond_m);
        while(next_work != NULL)
        {
            // There's still work, signal a worker to pick it up
            pthread_cond_broadcast(&my_cond);

            // Wait for work to be picked up
            pthread_cond_wait(&my_cond, &my_cond_m);
        }

        // No work is available now, let's put work on the queue
        next_work = create_work();
        printf("Producer: Created work %d\n", *next_work);

        pthread_mutex_unlock(&my_cond_m);
    }

    // Some workers might still be waiting, release them
    pthread_cond_broadcast(&my_cond);

    all_work_done = 1;
    return NULL;
}

int main()
{
    pthread_t t1, t2, t3, t4;

    pthread_create(&t1, NULL, worker, NULL);
    pthread_create(&t2, NULL, worker, NULL);
    pthread_create(&t3, NULL, worker, NULL);
    pthread_create(&t4, NULL, worker, NULL);

    producer(NULL);

    pthread_join(t1, NULL);
    pthread_join(t2, NULL);
    pthread_join(t3, NULL);
    pthread_join(t4, NULL);
    return 0;
}