Executing multiple commands using Popen.stdin

Question

I'd like to execute multiple commands in a standalone application launched from a python script, using pipes. The only way I could reliably pass the commands to the stdin of the program was using Popen.communicate but it closes the program after the command gets executed. If I use Popen.stdin.write than the command executes only 1 time out of 5 or so, it does not work reliable. What am I doing wrong?

To elaborate a bit :

I have an application that listens to stdin for commands and executes them line by line. I'd like to be able to run the application and pass various commands to it, based on the users interaction with a GUI. This is a simple test example:

import os, string
from subprocess import Popen, PIPE

command = "anApplication" 
process = Popen(command, shell=False, stderr=None, stdin=PIPE)

process.stdin.write("doSomething1\n")
process.stdin.flush()
process.stdin.write("doSomething2\n")
process.stdin.flush()

I'd expect to see the result of both commands but I don't get any response. (If I execute one of the Popen.write lines multiple times it occasionally works.)

And if I execute:

process.communicate("doSomething1")

it works perfectly but the application terminates.

Answer 1

What you are trying to do is called a "coprocess".

What you are almost certainly missing is a stdin.flush(). Also, be sure to include the final newline your coprocess probably requires after each command.

Note that implementing coprocesses can be tricky, because it requires synchronization between the parent and the child process. Specifically, you need a way to tell when the subprocess has finished producing the output for each command.

Also, look out for the potential deadlock that can arise if you set the subprocess stdout and stderr to different pipes: if the subprocess fills the stderr it will block until you read stderr. So, either use the same pipe for stdout and stderr, or use a separate thread to read stderr.

Answer 2

It sounds like your application is treating input from a pipe in a strange way. This means it won't get all of the commands you send until you close the pipe.

So the approach I would suggest is just to do this:

process.stdin.write("command1\n")
process.stdin.write("command2\n")
process.stdin.write("command3\n")
process.stdin.close()

It doesn't sound like your Python program is reading output from the application, so it shouldn't matter if you send the commands all at once like that.