Java threading question - listening to n error streams

Question

Hello all,

Let me say first that my experience with threading is pretty low.

I have an app that starts up several other Java jars via the Runtime.exec method. The problem is that the jars that are started need to be run concurrently, but in order to get at the error stream for the started jars you have to basically have a loop 'sitting and listening' until the process completes.

This is what I have now:

_processes.add( Runtime.getRuntime().exec( commandList.toArray( new String[ commandList.size() ] ) ) );
Thread thread = new Thread( new Runnable() {
    private final int _processNumber = _processes.size() - 1;
    public void run() {
        String streamData = _processNumber + " : ";
        streamData += "StdError [\r";
        BufferedReader bufferedReader =
                new BufferedReader( new InputStreamReader( _processes.get( _processNumber ).getErrorStream() ) );
        String line = null;
        try {
            while ( ( line = bufferedReader.readLine() ) != null ) {
                streamData += line + "\r";
            }
            bufferedReader.close();
            streamData += "]\r";
            LOG.error( streamData );
        }
        catch ( Exception exception ) {
            LOG.fatal( exception.getMessage() );
            exception.printStackTrace();
        }
    }
} );
thread.start();

Can anyone explain how to get the 'error stream listener threads' to work properly?

TIA

Answer 1

There is a very good tutorial here on how to use Runtime.exec(). Read through all of it but in particular take a look at page 4 where it explains how to use a 'StreamGobbler' running in a separate thread to consume both the std out and std err stream from the process being executed.

Basically, what you should be looking to implement in pseduo code is:

Runtime rt1 = Runtime.getRuntime().exec("my command")
new StreamGobbler(rt1.getOutputStream()).start()
new StreamGobbler(rt1.getErrorStream()).start()
//repeat for each external process (rt2, rt3 etc)
...
rt1.waitFor()
rt2.waitFor()
rt3.waitFor()

I.e. you kick off each process, and immediately start consuming each process's output in a separate thread. With the consuming threads started, just wait for each process to finish and return.

Answer 2

Instead of using Runtime.getRuntime().exec(), use Process to start external processes.. It'll make your life a whole lot easier..

Example code from a project of mine:

 //Build command 
 List<String> commands = new ArrayList<String>();
 commands.add("my_application");
 commands.add("arg1");
 commands.add("arg2");
 log.debug("{}", commands);

 //Run command with arguments
 ProcessBuilder pb = new ProcessBuilder(commands);
 pb.directory(directory);
 pb.redirectErrorStream(true);
 Process process = pb.start();

 //Read output
 StringBuilder out = new StringBuilder();
 BufferedReader br = new BufferedReader(new InputStreamReader
  (process.getInputStream()));

 //Only log unique lines (you might not need this)
 String line = null, previous = null;
 while ((line = br.readLine()) != null)
  if (!line.equals(previous)) {
   previous = line;
   out.append(line).append('\n');
   log.debug(line);
  }

 //Check result
 if (process.waitFor() == 0)
  return 0;

 //Abnormal termination: Log command parameters and output and throw ExecutionException
 log.error("{}", commands);
 log.error("\n{}", out.toString());
 throw new ExecutionException(new IllegalStateException("MyApplication exit code 1"));