How to get output from subprocess.Popen()

Question

I want output from execute Test_Pipe.py, I tried following code on Linux but not work.

Test_Pipe.py

import time
while True :
    print "Someting ..."
    time.sleep(.1)

Caller.py

import subprocess as subp
import time

proc = subp.Popen(["python", "Test_Pipe.py"], stdout=subp.PIPE, stdin=subp.PIPE)

while True :
    data = proc.stdout.readline() #block / wait
    print data
    time.sleep(.1)

The line proc.stdout.readline() was blocked, no data print out.

Help please, Thank you.

Answer 1

You obviously can use subprocess.communicate but I think you are looking for real time input and output.

readline was blocked because the process is probably waiting on your input. You can read character by character to overcome this like the following:

import subprocess
import sys

process = subprocess.Popen(
    cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE
)

while True:
    out = process.stdout.read(1)
    if out == '' and process.poll() != None:
        break
    if out != '':
        sys.stdout.write(out)
        sys.stdout.flush()

Answer 2

To avoid the many problems that can always arise with buffering for tasks such as "getting the subprocess's output to the main process in real time", I always recommend using pexpect for all non-Windows platform, wexpect on Windows, instead of subprocess, when such tasks are desired.

Answer 3

Nadia's snippet does work but calling read with a 1 byte buffer is highly unrecommended. The better way to do this would be to set the stdout file descriptor to nonblocking using fcntl

    fcntl.fcntl(
        proc.stdout.fileno(),
        fcntl.F_SETFL,
        fcntl.fcntl(proc.stdout.fileno(), fcntl.F_GETFL) | os.O_NONBLOCK,
    )

and then using select to test if the data is ready

    while True:
        readx = select.select([proc.stdout.fileno()], [], [])[0]
        if readx:
            chunk = proc.stdout.read()
            print chunk

for more info go here. She was correct in that your problem must be different from what you posted as Caller.py and Test_Pipe.py do work as provided.