programs running in parallel, read/writing in C

Question

I'm considering a set of 4 programs: (Prog1, Prog2, Prog3, Prog4) interacting with 4 files (FileA, FileB, FileC, FileD)

• Prog1: writes (appends) to FileA
• Prog2: reads File A and writes (appends) to FileB
• Prog3: reads File A, and writes (appends) to FileC
• Prog4: reads File B, and writes (appends) to FileD

or Potentially Prog1, might also read upon startup, and write continuously to say FileX.

Now all 4 programs will be running simultaneously (over network potentially but it shouldn't matter). Will this work?

Do I need to set "Strobes" or "busy" signals (i could do that with say mkdir, and rmdir)?

Answer 1

I think you need some kind of real FIFO structure here, also called pipes. There are constructs with that name under Windows and Unix-flavour OS'es.

An example under Linux can be found here, named pipes under Windows here

Answer 2

Is your problem that of synchronizing the reads/writes? Writes are the more problematic part since they modify the contents. Further, the nature of of the write (append at end, append at begining etc) may further complicate your situation. I have a feeling that you may need to look up "file locks"/mutexes etc. A lot depends on the OS(-es) you plan to run these on. Boost.Interprocess is a good place to start.

Answer 3

It can be made to work. You need to consider which process opens each file, and where the data that Prog1 is writing comes from.

If each program opens the files it works with, then there is no major problem. The main issue is the same as 'tail -f' has to deal with, namely that each of the reading processes is likely to read to EOF, and then has to pause and retry to see when more data becomes available.

If you have a central process that opens all the files, then you need to open file A for reading twice so that Prog2 and Prog3 have independent access to the file. However, it seems more sensible for any coordinator process to simply tell the children which files to open.

I don't see any need for strobes or busy signals. You've not warned of any 'near hard real time' response requirements or other special conditions that might warrant special programming.