Stopping a program if the next process in the pipeline was stopped.

Question

Hi all, when I'm running the following pipeline:

cat my_large_file.txt | head | wc

the process stops almost immediately. OK.

but when I run my java program

java MyProgramReadALargeFile my_large_file.txt | head | wc

the output from 'wc' is printed to stdout but the java program is still running. How can it detects that the pipeline was closed ?

Thanks,

Answer 1

'head' closes the stream after the first 10 lines, so it is normal that Java is still running while 'head' and 'wc' finished. Perhaps you can test in your Java code if System.out is still open, but I doubt this is a reliable method. It probably depends on the OS implementation of pipes and the JVM implementation of System.out (does it really close the stream or just passes it to /dev/null).

(have you tried cat'ing a file that is large enough and checking if the 'cat' process also keeps on running after 'head' is done?)

Answer 2

I doubt it closes System.out from the perspective of the java prog. It might redirect to /dev/null but would be easier for head to stop listening to the java program's output stream, i.e. exit().

You could look at the source for 'cat' to see if it does any detection on the state of the output stream. If that's the case, you'd have to put similar code in your 'MyProgramReadALargeFile' program. Otherwise head might be doing something the process feeding the pipe can't detect.

If the speed at which the 'cat' example finishes is suspicious, try doing

cat my_large_file.txt > /dev/null

and see if its just as fast.

Answer 3

Most UNIX utilities pay attention to SIGPIPE, which, when received, indicates that a piped stream to which the process is writing has nothing listening on the other end, so there's no need to keep working.

Searching for something like "java posix signals" yields some libraries that might allow you to add signal handlers to your Java code.