Do atomic operations work the same across processes as they do across threads?

Question

Obviously, atomic operations make sure that different threads don't clobber a value. But is this still true across processes, when using shared memory? Even if the processes happen to be scheduled by the OS to run on different cores? Or across different distinct CPUs?

Edit: Also, if it's not safe, is it not safe even on an operating system like Linux, where processes and threads are the same from the scheduler's point of view?

Answer 1

Atomic operations rely on special processor instructions, see Wikipedia. In other words, these operations and constructs built on top of them are not safe across process boundaries on multi-core systems. In that case you will have to implement some other forms of (probably slow) locking.

Answer 2

tl;dr: Read the fine print in the documentation of the atomic operations. Some will be atomic by design but may trip over certain variable types. In general, though, an atomic operation will maintain its contract between different processes just as it does between threads.

An atomic operation really only ensures that you won't have an inconsistent state if called by two entities simultaneously. For example, an atomic increment that is called by two different threads or processes on the same integer will always behave like so:

1. x = initial value (zero for the sake of this discussion)
2. Entity A increments x and returns the result to itself: result = x = 1.
3. Entity B increments x and returns the result to itself: result = x = 2.

where A and B indicate the first and second thread or process that makes the call.

A non-atomic operation can result in inconsistent or generally crazy results due to race conditions, incomplete writes to the address space, etc. For example, you can easily see this:

1. x = initial value = zero again.
2. Entity A calls x = x + 1. To evaluate x + 1, A checks the value of x (zero) and adds 1.
3. Entity B calls x = x + 1. To evaluate x + 1, B checks the value of x (still zero) and adds 1.
4. Entity B (by luck) finishes first and assigns the result of x + 1 = 1 (step 3) to x. x is now 1.
5. Entity A finishes second and assigns the result of x + 1 = 1 (step 2) to x. x is now 1.

Note the race condition as entity B races past A and completes the expression first.

Now imagine if x were a 64-bit double that is not ensured to have atomic assignments. In that case you could easily see something like this:

1. A 64 bit double x = 0.
2. Entity A tries to assign 0x1122334455667788 to x. The first 32 bits are assigned first, leaving x with 0x1122334400000000.
3. Entity B races in and assigns 0xffeeddccbbaa9988 to x. By chance, both 32 bit halves are updated and x is now = 0xffeeddccbbaa9988.
4. Entity A completes its assignment with the second half and x is now = 0xffeeddcc55667788.

These non-atomic assignments are some of the most hideous concurrent bugs you'll ever have to diagnose.