Multithreading: Read from / write to a pipe

Question

I write some data to a pipe - possibly lots of data and at random intervals. How to read the data from the pipe?

Is this ok:

• in the main thread (current process) create two more threads (2, 3)
• the second thread writes sometimes to the pipe (and flush-es the pipe?)
• the 3rd thread has infinite loop which reads the pipe (and then sleeps for some time)

Is this so far correct?

Now, there are a few thing I don't understand:

• do I have to lock (mutex?) the pipe on write?
• IIRC, when writing to pipe and its buffer gets full, the write end will block until I read the already written data, right? How to check for read data in the pipe, not too often, not too rarely? So that the second thread wont block? Is there something like select for pipes?
• It is possible to set the pipe to unbuffered more or I have to flush it regularly - which one is better?
• Should I create one more thread, just for flushing the pipe after write? Because flush blocks as well, when the buffer is full, right? I just don't want the 1st and 2nd thread to block....

[Edit] Sorry, I thought the question is platform agnostic but just in case: I'm looking at this from Win32 perspective, possibly MinGW C...

Answer 1

I'm not answering all of your questions here because there's a lot of them, but in answer to:

do I have to lock (mutex?) the pipe on write?

The answer to this question is platform specific, but in most cases I would guess yes.

It comes down to whether the write/read operations on the pipe are atomic. If either the read or write operation is non-atomic (most likely the write) then you will need to lock the pipe on writing and reading to prevent race conditions.

For example, lets say a write to the pipe compiles down to 2 instructions in machine code:

INSTRUCTION 1
INSTRUCTION 2

Let's say you get a thread context switch between these 2 instructions and your reading thread attempts to read the pipe which is in an intermediate state. This could result in a crash, or (worse) data corruption which can often manifest itself in a crash somewhere else in the code. This will often occur as a result of a race condition which are often non-deterministic and difficult to diagnose or reproduce.

In general, unless you can guarantee that all threads will be accessing the shared resource using an atomic instruction set, you must use mutexes or critical sections.