Communicate multiple times with a process without breaking the pipe?

Question

Hello, it's not the first time I'm having this problem and its really bugging me. Whenever I open a pipe using the Python subprocess module, I can only communicate with it once, as the documentation specifies: Read data from stdout and stderr, until end-of-file is reached

proc = sub.Popen("psql -h darwin -d main_db".split(),stdin=sub.PIPE,stdout=sub.PIPE)
print proc.communicate("select a,b,result from experiment_1412;\n")[0]
print proc.communicate("select theta,zeta,result from experiment_2099\n")[0]

The problem here is that the second time, Python isn't happy. Indeed, he decided to close the file after the first communicate:

Traceback (most recent call last):
File "a.py", line 30, in <module>
    print proc.communicate("select theta,zeta,result from experiment_2099\n")[0]
File "/usr/lib64/python2.5/subprocess.py", line 667, in communicate
    return self._communicate(input)
File "/usr/lib64/python2.5/subprocess.py", line 1124, in _communicate
     self.stdin.flush()
ValueError: I/O operation on closed file

So... multiple communications aren't allowed? I hope not ;)

Please enlighten me.

Answer 1

I've had this problem before, and as far as I could ever figure, you couldn't do this with subprocess (which, I agree, is very counterintuitive if true). I ended up using pexpect (obtainable from PyPI).

Answer 2

I think you misunderstand communicate...

http://docs.python.org/library/subprocess.html#subprocess.Popen.communicate

communicate sends a string to the other process and then waits on it to finish... (Like you said waits for the EOF listening to the stdout & stderror)

What you should do instead is:

proc.stdin.write('message')

# ...figure out how long or why you need to wait...

proc.stdin.write('message2')

(and if you need to get the stdout or stderr you'd use proc.stdout or proc.stderr)