Capturing Console output that is not written to stdout,stderr?

Question

Hi,

I have an windows application called pregeocode (we lack source), this program basically writes geocoding to an input file. This program doesn't actually write anything to console unless there is an error. This program is generally called from a small Python program (it handles the arguments etc, and does all the fun preprocessing).

We check if it fails by seeing if the output file was actually created (it always returns 0, no matter what). However when it fails subprocess shows that nothing was printed to stderr or stdout. (It processes around a hundred or so successfully, with ussally only a single one being bad, but i would love to be able to see what causes the error)

The small python script calls the application via subprocess.Popen:

argslist = [r'C:\workspace\apps\pregeocode.exe', '-in', inputfilename, '-out', outputfilename, '-gcp', gcp_file]
p = subprocess.Popen(argslist, stderr=subprocess.PIPE, stdout=subprocess.PIPE)
print str(p.communicate())

Gives the output of:

('', '')

However if i run the program manually using the same arguments via cmd, i get the output of:

45 IMAGE_EXTENT_TOO_SMALL

(There is around 60 odd different error messages, 45 indicates the error number)

Using the shell=True argument does not change anything, nor can i find anything online about this problem. The actual exe is something that was made in house a long time ago, and we lack the source code for it so i can't see how it prints out the messages.

So why can't subprocess actually capture the stdout or stderr of this?

EDIT

os.system(" ".join(argslist))

properly prints the error message:

45 IMAGE_EXTENT_TOO_SMALL

EDIT 2

Turns out the application uses ERDAS's toolkit. Their toolkit redirects all stdout/stderr into their logging subsystem. The logging subsystem then rewrites it via "CON".

Answer 1

Since the error message really isn't coming in on either stdout nor stderr, my best guess is that the program is using Windows' equivalent of opening /dev/tty, whatever that is. In Unix you could intercept that with careful use of pty.openpty but, as far as I know, there is no support for similar Windows-specific tricks in Python. You might try Expect for Windows instead.

Answer 2

Are you using python 2.5 or the early version of 2.6 ? Can you try this subprocess.check_output with stderr redirecting to stdout:

out_err = subprocess.check_output(argslist, stderr=subprocess.STDOUT)

If out_err gets both the output and error, you'll probably need to redirect stdout and stderr to file objects and later read from those file objects (because windows do not have unix-like pipes):

fout = open('fout', 'w')
ferr = open('ferr', 'w')    
p = subprocess.Popen(argslist, stdout=fout, stderr=ferr)
p.wait()
fout.close()
ferr.close()