Python: cannot read / write in another commandline application by using subprocess module

Question

I am using Python 3.0 in Windows and trying to automate the testing of a commandline application. The user can type commands in Application Under Test and it returns the output as 2 XML packets. One is a packet and the other one is an packet. By analyzing these packets I can verifyt he result. I ahev the code as below

p = subprocess.Popen(SomeCmdAppl, stdout=subprocess.PIPE,

                   shell = True, stdin=subprocess.PIPE, stderr=subprocess.STDOUT)

p.stdin.write((command + '\r\n').encode())
time.sleep(2.5)
testresult = p.stdout.readline()
testresult = testresult.decode()
print(testresult)

I cannot ge any output back. It get stuck in place where I try to read the output by using readline(). I tried read() and it get stuck too

When I run the commandline application manually and type the command I get the output back correctly as tow xml packets as below

Sent: <PivotNetMessage>
<MessageId>16f8addf-d366-4031-b3d3-5593efb9f7dd</MessageId>
<ConversationId>373323be-31dd-4858-a7f9-37d97e36eb36</ConversationId>
<SageId>4e1e7c04-4cea-49b2-8af1-64d0f348e621</SagaId>
<SourcePath>C:\Python30\PyNTEST</SourcePath>
<Command>echo</Command>
<Content>Hello</Content>
<Time>7/4/2009 11:16:41 PM</Time>
<ErrorCode>0</ErrorCode>
<ErrorInfo></ErrorInfo>
</PivotNetMessagSent>

Recv: <PivotNetMessage>
<MessageId>16f8addf-d366-4031-b3d3-5593efb9f7dd</MessageId>
<ConversationId>373323be-31dd-4858-a7f9-37d97e36eb36</ConversationId>
<SageId>4e1e7c04-4cea-49b2-8af1-64d0f348e621</SagaId>
<SourcePath>C:\PivotNet\Endpoints\Pipeline\Pipeline_2.0.0.202</SourcePath>
<Command>echo</Command>
<Content>Hello</Content>
<Time>7/4/2009 11:16:41 PM</Time>
<ErrorCode>0</ErrorCode>
<ErrorInfo></ErrorInfo>
</PivotNetMessage>

But when I use the communicate() as below I get the Sent packet and never get the Recv: packet. Why am I missing the recv packet? The communicate(0 is supposed to bring everything from stdout. rt?

p = subprocess.Popen(SomeCmdAppl, stdout=subprocess.PIPE,

                   shell = True, stdin=subprocess.PIPE, stderr=subprocess.STDOUT)
p.stdin.write((command + '\r\n').encode())
time.sleep(2.5)
result = p.communicate()[0]
print(result)

Can anybody help me with a sample code that should work? I don't know if it is needed to read and write in separate threads. Please help me. I need to do repeated read/write. Is there any advanced level module in python i can use. I think Pexpect module doesn't work in Windows

Answer 1

Try sending your input using communicate instead of using write:

result = p.communicate((command + '\r\n').encode())[0]

Answer 2

This is a popular problem, e.g. see:

• http://stackoverflow.com/questions/1124884/interact-with-a-windows-console-application-via-python
• http://stackoverflow.com/questions/874815/how-do-i-get-real-time-information-back-from-a-subprocess-popen-in-python-2-5
• http://stackoverflow.com/questions/1161580/how-do-i-read-everything-currently-in-a-subprocess-stdout-pipe-and-then-return

(Actually, you should have seen these during creation of your question...?!).

I have two things of interest:

• p.stdin.write((command + '\r\n').encode()) is also buffered so your child process might not even have seen its input. You can try flushing this pipe.
• In one of the other questions one suggested doing a stdout.read() on the child instead of readline(), with a suitable amount of characters to read. You might want to experiment with this.

Post your results.

Answer 3

Have you considered using pexpect instead of subprocess? It handles the details which are probably preventing your code from working (like flushing buffers, etc). It may not be available for Py3k yet, but it works well in 2.x.

See: http://pexpect.sourceforge.net/pexpect.html