Logging output of external program with (wx)python

Question

Hi

I'm writing a GUI for using the oracle exp/imp commands and starting sql-scripts through sqlplus. The subprocess class makes it easy to launch the commands, but I need some additional functionality. I want to get rid of the command prompt when using my wxPython GUI, but I still need a way to show the output of the exp/imp commands.

I already tried these two methods:

command = "exp userid=user/pwd@nsn file=dump.dmp"

process = subprocess.Popen(command, stdout=subprocess.PIPE)
output = process.communicate()[0]

process = subprocess.Popen(command, stdout=subprocess.PIPE)
process.wait()
output = process.stdout.read()

Through on of these methods (forgot which one) I really got the output of exp/imp, but only after the command finishes, which is quite worthless to me, as I need a frequent update during these potentially long running operations. And sqlplus made even more problems, as sqlplus mostly wants some input when an error occurs. When this happens python waits for the process to finish but the user can't see the prompt, so you don't know how long to wait or what to do...

What I'd like to have is a wrapper that outputs everything I can see on the standard commandline. I want to log this to a file and show it inside a wxPython control.

I also tried the code from this page: http://code.activestate.com/recipes/440554/ but this can't read the output either. The OutputWrapper from this answer doesn't work either: http://stackoverflow.com/questions/166198/how-can-i-capture-all-exceptions-from-a-wxpython-application

Any help would be appreciated!

EDIT:
The subprocesses don't seem to flush their output. I already tried it with .readline().
My Tool has to run on windows and unix, so pexpect is no solution if there's no windows version. And using cx_oracle would be extreme overkill as I would have to rebuild the whole functionality of exp, imp and sqlplus.

Answer 1

Try this:

import subprocess

command = "ping google.com"

process = subprocess.Popen(command, shell=True, stdout=subprocess.PIPE)
output = process.stdout
while 1:
    print output.readline(),

Answer 2

The solution is to use a list for your command

command = ["exp", "userid=user/pwd@nsn", "file=dump.dmp"]
process = subprocess.Popen(command, stdout=subprocess.PIPE)

then you read process.stdout in a line-by-line basis:

line = process.stdout.readline()

that way you can update the GUI without waiting. IF the subprocess you are running (exp) flushes output. It is possible that the output is buffered, then you won't see anything until the output buffer is full. If that is the case then you are probably out of luck.

Answer 3

If you're on Linux, check out pexpect. It does exactly what you want.

If you need to work on Windows, maybe you should bite the bullet and use Python bindings to Oracle, such as cx_Oracle, instead of running CL stuff via subprocess.

Answer 4

Are these solutions able to capture stderr as well? I see you have stdout= option above. How do you make sure to get stderr as well? Another question is is there a way to use import logging/import logging.handlers to capture command stdout/stderr. It would be interesting to be able to use the logger with its buildt in formatters/rotaters,etc.