How to capture Python interpreter's and/or CMD.EXE's output from a Python script?

Question

1. Is it possible to capture Python interpreter's output from a Python script?
2. Is it possible to capture Windows CMD's output from a Python script?

If so, which librar(y|ies) should I look into?

Answer 1

In which context are you asking?

Are you trying to capture the output from a program you start on the command line?

if so, then this is how to execute it:

somescript.py | your-capture-program-here

and to read the output, just read from standard input.

If, on the other hand, you're executing that script or cmd.exe or similar from within your program, and want to wait until the script/program has finished, and capture all its output, then you need to look at the library calls you use to start that external program, most likely there is a way to ask it to give you some way to read the output and wait for completion.

Answer 2

You want subprocess. Look specifically at Popen in 17.1.1 and communicate in 17.1.2.

Answer 3

If you are talking about the python interpreter or CMD.exe that is the 'parent' of your script then no, it isn't possible. In every POSIX-like system (now you're running Windows, it seems, and that might have some quirk I don't know about, YMMV) each process has three streams, standard input, standard output and standard error. Bu default (when running in a console) these are directed to the console, but redirection is possible using the pipe notation:

python script_a.py | python script_b.py

This ties the standard output stream of script a to the standard input stream of script B. Standard error still goes to the console in this example. See the article on standard streams on Wikipedia.

If you're talking about a child process, you can launch it from python like so (stdin is also an option if you want two way communication):

import subprocess
# Of course you can open things other than python here :)
process = subprocess.Popen(["python", "main.py"], stdout=subprocess.PIPE, stderr=subprocess.PIPE)
x = process.stderr.readline()
y = process.stdout.readline()
process.wait()

See the Python subprocess module for information on managing the process. For communication, the process.stdin and process.stdout pipes are considered standard file objects.

For use with pipes, reading from standard input as lassevk suggested you'd do something like this:

import sys
x = sys.stderr.readline()
y = sys.stdin.readline()

sys.stdin and sys.stdout are standard file objects as noted above, defined in the sys module. You might also want to take a look at the pipes module.

Reading data with readline() as in my example is a pretty naïve way of getting data though. If the output is not line-oriented or indeterministic you probably want to look into polling which unfortunately does not work in windows, but I'm sure there's some alternative out there.

Answer 4

Actually, you definitely can, and it's beautiful, ugly, and crazy at the same time!

You can replace sys.stdout and sys.stderr with StringIO objects that collect the output.

Here's an example, save it as evil.py:

import sys
import StringIO

s = StringIO.StringIO()

sys.stdout = s

print "hey, this isn't going to stdout at all!"
print "where is it ?"

sys.stderr.write('It actually went to a StringIO object, I will show you now:\n')
sys.stderr.write(s.getvalue())

When you run this program, you will see that:

• nothing went to stdout (where print usually prints to)
• the first string that gets written to stderr is the one starting with 'It'
• the next two lines are the ones that were collected in the StringIO object

Replacing sys.stdout/err like this is an application of what's called monkeypatching. Opinions may vary whether or not this is 'supported', and it is definitely an ugly hack, but it has saved my bacon when trying to wrap around external stuff once or twice.

Tested on Linux, not on Windows, but it should work just as well. Let me know if it works on Windows!

Answer 5

I think I can point you to a good answer for the first part of your question.

1.  Is it possible to capture Python interpreter's output from a Python script?

The answer is "yes", and personally I like the following lifted from the examples in the PEP 343 -- The "with" Statement document.

from contextlib import contextmanager

@contextmanager
def opened(filename, mode="r"):
    f = open(filename, mode)
    try:
        yield f
    finally:
        f.close()

@contextmanager
def stdout_redirected(new_stdout):
    save_stdout = sys.stdout
    sys.stdout = new_stdout
    try:
        yield None
    finally:
        sys.stdout = save_stdout

And used like this:

with opened(filename, "w") as f:
    with stdout_redirected(f):
        print "Hello world"

Which could, of course, be extended to also redirect sys.stderr to the same file. A nice aspect of it is that it can be applied selectively around just a portion of a script's execution, rather than its entire extent, and stays in effect even when unhandled exceptions are raised within its context.