messed up using do_futex?

Question

I'm getting a weird error. I implemented these two functions:

int flag_and_sleep(volatile unsigned int *flag)
{
    int res = 0;

    (*flag) = 1;

    res = syscall(__NR_futex, flag, FUTEX_WAIT, 1, NULL, NULL, 0);
    if(0 == res && (0 != (*flag)))
        die("0 == res && (0 != (*flag))");
    return 0;
}

int wake_up_if_any(volatile unsigned int *flag)
{
    if(1 == (*flag))
    {
        (*flag) = 0;
        return syscall(__NR_futex, flag, FUTEX_WAKE, 1, NULL, NULL, 0);
    }
    return 0;
}

and test them by running two Posix threads:

static void die(const char *msg)
{
    fprintf(stderr, "%s %u %lu %lu\n", msg, thread1_waits, thread1_count, thread2_count);
    _exit( 1 );
}

volatile unsigned int thread1_waits = 0;

void* threadf1(void *p)
{
    int res = 0;
    while( 1 )
    {
        res = flag_and_sleep( &thread1_waits );
        thread1_count++;
    }
    return NULL;
}

void* threadf2(void *p)
{
    int res = 0;
    while( 1 )
    {
        res = wake_up_if_any( &thread1_waits );
        thread2_count++;
    }

    return NULL;
}

After thread2 has had a million or so iterations, I get the assert fire on me:

./a.out 0 == res && (0 != (*flag)) 1 261129 1094433

This means that the syscall - and thereby do_futex() - returned 0. Man says it should only do so if woken up by a do_futex(WAKE) call. But then before I do a WAKE call, I set the flag to 0. Here it appears that flag is still 1.

This is Intel, which means strong memory model. So if in thread1 I see results from a syscall in thread2, I must also see the results of the write in thread 2 which was before the call.

Flag and all pointers to it are volatile, so I don't see how gcc could fail to read the correct value.

I'm baffled.

Thanks!