How do I run a sub-process, display its output in a GUI and allow it to be terminated?

Question

I have been trying to write an application that runs subprocesses and (among other things) displays their output in a GUI and allows the user to click a button to cancel them. I start the processes like this:

queue = Queue.Queue(500)
process = subprocess.Popen(
    command,
    stdout=subprocess.PIPE,
    stderr=subprocess.STDOUT)
iothread = threading.Thread(
    target=simple_io_thread,
    args=(process.stdout, queue))
iothread.daemon=True
iothread.start()

where simple_io_thread is defined as follows:

def simple_io_thread(pipe, queue):
    while True:
        line = pipe.readline()
        queue.put(line, block=True)
        if line=="":
            break

This works well enough. In my UI I periodically do non-blocking "get"s from the queue. However, my problems come when I want to terminate the subprocess. (The subprocess is an arbitrary process, not something I wrote myself.) I can use the terminate method to terminate the process, but I do not know how to guarantee that my I/O thread will terminate. It will normally be doing blocking I/O on the pipe. This may or may not end some time after I terminate the process. (If the subprocess has spawned another subprocess, I can kill the first subprocess, but the second one will still keep the pipe open. I'm not even sure how to get such grand-children to terminate cleanly.) After that the I/O thread will try to enqueue the output, but I don't want to commit to reading from the queue indefinitely.

Ideally I would like some way to request termination of the subprocess, block for a short (<0.5s) amount of time and after that be guaranteed that the I/O thread has exited (or will exit in a timely fashion without interfering with anything else) and that I can stop reading from the queue.

It's not critical to me that a solution uses an I/O thread. If there's another way to do this that works on Windows and Linux with Python 2.6 and a Tkinter GUI that would be fine.

---

EDIT - Will's answer and other things I've seen on the web about doing this in other languages suggest that the operating system expects you just to close the file handle on the main thread and then the I/O thread should come out of its blocking read. However, as I described in the comment, that doesn't seem to work for me. If I do this on the main thread:

process.stdout.close()

I get:

IOError: close() called during concurrent operation on the same file object.

...on the main thread. If I do this on the main thread:

os.close(process.stdout.fileno())

I get:

close failed in file object destructor: IOError: [Errno 9] Bad file descriptor

...later on in the main thread when it tries to close the file handle itself.

Answer 1

In the code that terminates the process, you could also explicitly os.close() the pipe that your thread is reading from?