Data in linux FIFO seems lost

Answer 1

I get all four lines in the log file when I run it. What happens if you change your shebang to #!/bin/bash?

Answer 2

It could be a concurrency issue, with both subshells trying to read from the same fifo at the same time. Does it happen all the time?

You could try adding a flock -x 6 statement or change the delay for the two subshells and see what happens.

BTW, I can confirm that with bash 3.2 and kernel 2.6.28 your code works fine.

Answer 3

Is it possible there is some buffering going on of your write to the fifo? If you have unbuffer available, could you try prefacing the echos with that? I don't really see how it could happen here but the symptoms fit so its worth a shot.

Answer 4

Keep in mind that a FIFO on POSIX systems is essentially a named pipe. In order to move data around on a pipe, one side needs a reader and the other side needs a writer, and when one is closed the other loses usefulness.

In other words, you cannot cat on a fifo after some other reader has exited, because the contents of the FIFO will be gone.

You may want to see about using a normal file (and use file locking to ensure that you are synchronizing your access to that normal file), or use a directory with multiple files in it, or even use shared memory or something similar to that (perhaps not in a shell script, though). It all depends on what your end-goal is, really, what the best way to go about it would be.

Answer 5

For reasons explained in other answers here you do not want a pipe unless you can read and write from the pipe at the same time.

It is therefore advisable to use another means of IPC or restructure your usage of fifos such that an asynchronous process fills up the pipe while the main process creates work processes (or the other way around).

Here's a method of getting what you want using a simple file as a sort of queue:

#!/usr/bin/env bash

stack=/tmp/stack
> "$stack"

# Create an initial 5 spots on the stack
for i in {1..5}; do
    echo >> "$stack"
done

for i in {1..10}; do
    # Wait for a spot on the stack.
    until read; do sleep 1; done

    {
        echo "Starting process #$i"
        sleep $((5 + $i)) # Do something productive
        echo "Ending process #$i"

        # We're done, free our spot on the stack.
        echo >> "$stack"
    } &
done < "$stack"

Sidenote: This method isn't ideal for unlimited work since it adds a byte to the stack file for each process it invokes meaning the stack file grows slowly.