Python persistent Popen

Question

Is there a way to do multiple calls in the same "session" in Popen? For instance, can I make a call through it and then another one after it without having to concatenate the commands into one long string?

Answer 1

For instance, can I make a call through it and then another one after it without having to concatenate the commands into one long string?

Sounds like you're using shell=True. Don't, unless you need to. Instead use shell=False (the default) and pass in a command/arg list.

Is there a way to do multiple calls in the same "session" in Popen? For instance, can I make a call through it and then another one after it without having to concatenate the commands into one long string?

Any reason you can't just create two Popen instances and wait/communicate on each as necessary? That's the normal way to do it, if I understand you correctly.

Answer 2

You're not "making a call" when you use popen, you're running an executable and talking to it over stdin, stdout, and stderr. If the executable has some way of doing a "session" of work (for instance, by reading lines from stdin) then, yes, you can do it. Otherwise, you'll need to exec multiple times.

subprocess.Popen is (mostly) just a wrapper around execvp(3)

Answer 3

Assuming you want to be able to run a shell and send it multiple commands (and read their output), it appears you can do something like this:

from subprocess import *
p = Popen(['/bin/sh'], shell=False, stdin=PIPE, stdout=PIPE, stderr=PIPE)

After which, e.g.,:

>>> p.stdin.write("cat /etc/motd\n")
>>> p.stdout.readline()
'Welcome to dev-linux.mongo.com.\n'

(Of course, you should check stderr too, or else ask Popen to merge it with stdout). One major problem with the above is that the stdin and stdout pipes are in blocking mode, so it's easy to get "stuck" waiting forever for output from the shell. Although I haven't tried it, there's a recipe at the ActiveState site that shows how to address this.

Update: after looking at the related questions/answers, it looks like it might be simpler to just use Python's built-in select module to see if there's data to read on stdout (you should also do the same for stderr, of course), e.g.:

>>> select.select([p.stdout], [], [], 0)
([<open file '<fdopen>', mode 'rb' at 0x10341690>], [], [])