Operations from atomic.h seem to be non-atomic

Question

The following code produces random values for both n and v. It's not surprising that n is random without being properly protected. But it is supposed that v should finally be 0. Is there anything wrong in my code? Or could anyone explain this for me? Thanks.

I'm working on a 4-core server of x86 architecture. The uname is as follows.

Linux 2.6.9-22.ELsmp #1 SMP Mon Sep 19 18:00:54 EDT 2005 x86_64 x86_64 x86_64 GNU/Linux

#include <stdio.h>
#include <pthread.h>
#include <asm-x86_64/atomic.h>

int n = 0;
atomic_t v;
pthread_mutex_t mutex = PTHREAD_MUTEX_INITIALIZER;

#define LOOP 10000

void* foo(void *p)
{
    int i = 0;
    for(i = 0; i < LOOP; i++) {
//        pthread_mutex_lock(&mutex);
        ++n;
        --n;
        atomic_inc(&v);
        atomic_dec(&v);
//        pthread_mutex_unlock(&mutex);
    }

    return NULL;
}

#define COUNT 50

int main(int argc, char **argv)
{
    int i;
    pthread_t pids[COUNT];
    pthread_attr_t attr;
    pthread_attr_init(&attr);
    atomic_set(&v, 0);

    for(i = 0; i < COUNT; i++) {
        pthread_create(&pids[i], &attr, foo, NULL);
    }

    for(i = 0; i < COUNT; i++) {
        pthread_join(pids[i], NULL);
    }

    printf("%d\n", n);
    printf("%d\n", v);
    return 0;
}

Answer 1

This old post implies that

• It's not obvious that you're supposed to include this kernel header in userspace programs
• It's been known to fail to provide atomicity for userspace programs.

So ... Perhaps that's the reason for the problems you're seeing?

Answer 2

Can we get a look at the assembler output of the code (gcc -E, I think). Even thought the uname indicates it's SMP-aware, that doesn't necessarily mean it was compiled with CONFIG_SMP.

Without that, the assembler code output does not have the lock prefix and you can find your cores interfering with one another.

But I would be using the pthread functions anyway since they're portable across more platforms.

Answer 3

You should use gcc built-ins instead (see. this) This works fine, and also works with icc.

int a; 
__sync_fetch_and_add(&a, 1); // atomic a++

Note that you should be aware of the cache consistency issues when you modify variables without locking.