timeout a subprocess

Question

Hi,

I realize this might be a duplicate of http://stackoverflow.com/questions/1191374/subprocess-with-timeout. If it is, I apologize, just wanted to clarify something.

I'm creating a subprocess, which I want to run for a certain amount of time, and if it doesn't complete within that time, I want it to throw an error. Would something along the lines of the following code work or do we have to use a signal like answered in the other question? Thanks in advance!:

def run(self):
    self.runTestCmd()
    self.waitTestComplete(self.timeout)

def runTestCmd(self):
    self.proc = subprocess.Popen("./configure", shell=True)

def waitTestComplete(self, timeout):
    st = time.time() 
    while (time.time()-st) < timeout:
        if self.proc.poll() == 0:
            return True
        else:
            time.sleep(2)
    raise TestError("timed out waiting for test to complete")

Answer 1

It would, but it has a problem. The process will continue on doing whatever it is you asked it to do even after you've given up on it. You'll have to send the process a signal to kill it once you've given up on it if you really want it to stop.

Since you are spawning a new process (./configure which is presumably a configure script) that in turn creates a whole ton of sub-processes this is going to get a little more complex.

import os

def runTestCmd(self):
    self.proc = subprocess.Popen(["./configure"], shell=False,
                                 preexec_fn=os.setsid)

Then os.kill(-process.pid, signal.SIGKILL) should kill all the sub-processes. Basically what you are doing is using the preexec_fn to cause your new subprocess to acquire it's own session group. Then you are sending a signal to all processes in that session group.

Many processes that spawn subprocesses know that they need to clean up their subprocesses before they die. So it behooves you to try being nice to them if you can. Try os.signal(-process.pid, signal.SIGTERM) first, wait a second or two for the process to exit, then try SIGKILL. Something like this:

import time, os, errno, signal

def waitTestComplete(self, timeout):
    st = time.time() 
    while (time.time()-st) < timeout:
        if self.proc.poll() is not None:  # 0 just means successful exit
            # Only return True if process exited successfully,
            # otherwise return False.
            return self.proc.returncode == 0
        else:
            time.sleep(2)
    # The process may exit between the time we check and the
    # time we send the signal.
    try:
        os.kill(-self.proc.pid, signal.SIGTERM)
    except OSError, e:
        if e.errno != errno.ESRCH:
            # If it's not because the process no longer exists,
            # something weird is wrong.
            raise
    time.sleep(1)
    if self.proc.poll() is None: # Still hasn't exited.
        try:
            os.kill(-self.proc.pid, signal.SIGKILL)
        except OSError, e:
            if e.errno != errno.ESRCH:
                raise
    raise TestError("timed out waiting for test to complete")

As a side note, never, ever use shell=True unless you know for absolute certain that's what you want. Seriously. shell=True is downright dangerous and the source of many security issues and mysterious behavior.