STL Structures in Static Memory 'Losing' Data Across Threads

Question

Hey all -

I'm writing a multi-threaded demo program using pthreads, where one thread loads data into an STL queue, and another thread reads from it. Sounds trivial, right? Unfortunately, data pushed into the queue is vanishing. I'm not new to multithreading, nor am I unfamiliar with memory structures - however, this has me stumped.

These are my declarations for the queue itself and the mutex that protects it, which are located in a header included by the client code:

static std::queue<int32_t> messageQueue;
static pthread_mutex_t messageQueueLock; 

When the program starts up, it initializes the mutex with the process shared attribute:

pthread_mutexattr_t sharedAttr;
pthread_mutexattr_init(&sharedAttr);
pthread_mutexattr_setpshared(&sharedAttr, PTHREAD_PROCESS_SHARED);
pthread_mutex_init(&messageQueueLock, &sharedAttr);

It then launches the 'producer' thread and 'consumer' threads, and lets them do their thing. The producer thread pushes a new item onto the queue and then goes to sleep. Here are the lines where it adds something to the queue:

pthread_mutex_lock(&messageQueueLock);
messageQueue.push(message);
pthread_mutex_unlock(&messageQueueLock);

It then sleeps and lets the consumer thread take over. However, when the consumer thread checks for items in the queue, the queue is magically empty.

I have stepped through the program using gdb. Below is the output of my run. You can see where the producer adds something to the queue, I print the queue's size to make sure it's there, there is a context switch to the consumer thread, I print the queue's size again, and it's empty. Check it out:

(gdb) b main_ex.cpp:70
Breakpoint 1 at 0x100006a24: file main_ex.cpp, line 70.
(gdb) run
Starting program: a.out 
Reading symbols for shared libraries ++. done
Creating the mutex.
Producer thread starting up. 
PRODUCER: Creating a message to send.
PRODUCER: Adding the message to the queue.
[Switching to process 7432]

Breakpoint 1, yourProcess () at main_ex.cpp:70
70      pthread_mutex_lock(&messageQueueLock);
(gdb) n
71      messageQueue.push(message);
(gdb) p messageQueue.size()
$1 = 0
(gdb) n
72      pthread_mutex_unlock(&messageQueueLock);
(gdb) p messageQueue.size()
$2 = 1
(gdb) b consumer.cpp:81
Breakpoint 2 at 0x1000043f7: file consumer.cpp, line 81.
(gdb) c
Continuing.
PRODUCER: Sleep time!
[Switching to process 7432]

Breakpoint 2, Producer::processMessageQueue (this=0x1001000c0) at producer.cpp:81
81      pthread_mutex_lock(&messageQueueLock);
(gdb) n
83      if(messageQueue.empty()) {
(gdb) p messageQueue.size()
$3 = 0
(gdb) quit

So, I'm really not sure what is going on. The queue is only ever accessed in a critical section (both read/write), the queue is static, and the header is if-def'd to not be multi-included.

I'm grateful for any help anyone can offer!

Answer 1

Header guards protect from multiple inclusion per translation unit. However, different translation units will re-include them.

In your case, it seems they are each getting their very own static queue and mutex. Also, consider even if you were correct: Without re-including the header, the translation unit would have no idea what queue and mutex are! You'd be trying to use some undeclared identifier.

You need extern, which is actually the opposite of static:

extern std::queue<int32_t> messageQueue;
extern pthread_mutex_t messageQueueLock;

Then in one unit, actually define them:

std::queue<int32_t> messageQueue;
pthread_mutex_t messageQueueLock;

Answer 2

You should verify whether the two threads are actually accessing the same queue or not, if not try to avoid the static queue and create it for example in the main function or somewhere convenient.