Interact with a Windows console application via Python

Question

Hello,

I am using python 2.5 on Windows. I wish to interact with a console process via Popen. I currently have this small snippit of code:

p = Popen( ["console_app.exe"], stdin=PIPE, stdout=PIPE )
# issue command 1...
p.stdin.write( 'command1\n' )
result1 = p.stdout.read() # <---- we never return here
# issue command 2...
p.stdin.write( 'command2\n' )
result2 = p.stdout.read()

I can successfully write to stdin but can not get anything back from stdout. Have I missed some step? I don't want to use p.communicate( "command" )[0] as it terminates the process and I need to interact with the process dynamically over a period of time.

Thanks in advance.

Answer 1

I think you might want to try to use readline() instead?

Edit: sorry, misunderstoud.

Maybe this question can help you?

Answer 2

Is it possible that the console app is buffering its output in some way so that it is only being sent to stdout when the pipe is closed? If you have access to the code for the console app, maybe sticking a flush after a batch of output data might help?

Alternatively, is it actually writing to stderr and instead of stdout for some reason?

Just looked at your code again and thought of something else, I see you're sending in "command\n". Could the console app be simply waiting for a carriage return character instead of a new line? Maybe the console app is waiting for you to submit the command before it produces any output.

Answer 3

Had the exact same problem here. I dug into DrPython source code and stole wx.Execute() solution, which is working fine, especially if your script is already using wx. I never found correct solution on windows platform though...

Answer 4

Your problem here is that you are trying to control an interactive application.

stdout.read() will continue reading until it has reached the end of the stream, file or pipe. Unfortunately, in case of an interactive program, the pipe is only closed then whe program exits; which is never, if the command you sent it was anything other than "quit".

You will have to revert to reading the output of the subprocess line-by-line using stdout.readline(), and you'd better have a way to tell when the program is ready to accept a command, and when the command you issued to the program is finished and you can supply a new one. In case of a program like cmd.exe, even readline() won't suffice as the line that indicates a new command can be sent is not terminated by a newline, so will have to analyze the output byte-by-byte. Here's a sample script that runs cmd.exe, looks for the prompt, then issues a dir and then an exit:

from subprocess import *
import re

class InteractiveCommand:
    def __init__(self, process, prompt):
        self.process = process
        self.prompt  = prompt
        self.output  = ""
        self.wait_for_prompt()

    def wait_for_prompt(self):
        while not self.prompt.search(self.output):
            c = self.process.stdout.read(1)
            if c == "":
                break
            self.output += c

        # Now we're at a prompt; clear the output buffer and return its contents
        tmp = self.output
        self.output = ""
        return tmp

    def command(self, command):
        self.process.stdin.write(command + "\n")
        return self.wait_for_prompt()

p      = Popen( ["cmd.exe"], stdin=PIPE, stdout=PIPE )
prompt = re.compile(r"^C:\\.*>", re.M)
cmd    = InteractiveCommand(p, prompt)

listing = cmd.command("dir")
cmd.command("exit")

print listing

If the timing isn't important, and interactivity for a user isn't required, it can be a lot simpler just to batch up the calls:

from subprocess import *

p = Popen( ["cmd.exe"], stdin=PIPE, stdout=PIPE )
p.stdin.write("dir\n")
p.stdin.write("exit\n")

print p.stdout.read()

Answer 5

Have you tried to force windows end lines? i.e.

p.stdin.write( 'command1 \r\n' )
p.stdout.readline()

UPDATE:

I've just checked the solution on windows cmd.exe and it works with readline(). But it has one problem Popen's stdout.readline blocks. So if the app will ever return something without endline your app will stuck forever.

But there is a work around for that check out: http://code.activestate.com/recipes/440554/