Controlling access to output in multi-threaded applications

Question

I have an application that creates a job queue, and then multiple threads execute the jobs. By execute them, I mean they call system() with the job string.

The problem is that output to stdout looks like the output at the bottom of the question. I would like each application run to be separated, so the output would look like:

flac 1.2.1 ...
...
...

flac 1.2.1 ...
...
...

etc.

I'm using programs I have no control over, and so cannot wrap the IO in mutexes.

How do I get the output to look like the above?

ffllaacc  11..22..11,,  CCooppyyrriigghhtt  ((CC)) 2000,2001,2002, 2003,220004,2
005,0200,0260,0210,0270 0 2J,o2s0h0 3C,o2a004,2005,2l0s0o6n,
007f l aJco scho mCeosa lwsiotnh
 AfBl
OcL UcfTolEmaLecYs   1Nw.Oi2t .hW1 A,AR BRCSAoONpLTyUYrT.iE gL hYTt h Ni(OsC  )W
i AsR2 R0fA0rN0eT,eY2 .0s 0o 1fT,th2wi0as0r 2ei,,s2  0af0nr3de, e2y 0os0uo4 f,at
2rw0ea0
e,,2 0wa0en6ld,c 2oy0mo0eu7   ta orJ eor
hd iCswotearllicsbooumnte
 tiotf  lruaencdd iecsrot mrceiesbr utwtaieit nhi  tcA oBunSndOdiLetUriT oEcnLes
Yr. t Na OiT nyW pAceRo Rn`AdfNilTtaYic.o' n  sfT.oh ri  sTd yeiptsea  if`lrfsel
.ea
s
'f tfwor adreet,a iandl sy.ou
a
e
welcome to redistribute it under certain conditions.  Type `flac' for details.

Answer 1

Rather than using system().. you could use popen(). Then, read from each child's output in the parent program, and do what you want with it (e.g. synchronize on some mutex when outputting each line).

Answer 2

As pnm points out, popen() is probably the best way to go. You still have to store everything you read through the pipe and at the end write it to wherever you plan to keep it without interspersing it with the output of the other popen() calls.

An uglier, less efficient but possibly simpler way is to set up your system() jobs to redirect their output to files. You can then have a thread that gets passed the file names after the system() calls complete which appends the file contents in their entirety to your output and deletes the temporary files.

Or some variant of both. Use popen(), store the results of the pipe reads somewhere in memory, pass the memory address to the output thread when the popen calls complete, etc.

Answer 3

Besides what pnm and Duck wrote, you should check the man page for the externel programs, and check how they allow you to suppress unnecessary output.

And on a last resort you may suppress output through the shell, e.g.

system("flac ... > /dev/null"); or even stdout and stderr: system("flac ... 2>&1");

Answer 4

I think popen() is the best solution, but it may take some work.

I quick solution may be to re-direct the output of each application to a tmp file.
Then once all the threads are finished copy the tmp files to the standard output.

system("plop > tmp1");