Recover process with subprocess.Popen?

Question

Hi everyone!

I have a python program that uses subprocess.Popen to launch another process (python process or whatever), and after launching it I save the child's PID to a file. Let's suppose that suddenly the parent process dies (because of an exception or whatever). Is there any way to access again to the object returned by Popen?

I mean, the basic idea is to read the file at first, and if it exists and it has a PID written on it, then access to that process someway, in order to know the return code or whatever. If there isn't a PID, then launch the process with Popen.

Thanks a lot!!

Answer 1

Looks like you're trying to write a daemon, and it needs pidfile support. You can't go wrong with python-daemon.

For example:

import daemon
import lockfile
import os

with daemon.DaemonContext(pidfile=lockfile.FileLock('/var/run/spam.pid')):
    os.execl('/path/to/prog', args…)

Answer 2

The Popen object is effectively just a wrapper for the child processes PID, stdin, stdout, and stderr, plus some convenience functions for using those.

So the question is why do you need access to the Popen object? Do you want to communicate with the child, terminate it, or check whether it's still running?

In any case there is no way reacquire a Popen object for an already running process.

The proper way to approach this is to launch the child as a daemon, like Tobu suggested. Part of the procedure for daemonising a process is to close stdin and stdout, so you cannot use those to talk to the child process. Instead most daemons use either pipes or sockets to allow clients to connect to them and to send them messages.

The easiest way to talk to the child is to open a named pipe from the child process at e.g. /etc/my_pipe, open that named pipe from the parent / controlling process, and write / read to / from it.

After a quick look at python-daemon it seems to me that python-daemon will help you daemonise your child process, which is tricky to get right, but it doesn't help you with the messaging side of things.

But like I said, I think you need to tell us why you need a Popen object for the child process before we can help you any further.

Answer 3

If a process dies, all its open file handles are closed. This includes any unnamed pipes created by popen(). So, no, there's no way to recover a Popen object from just a PID. The OS won't even consider your new process the parent, so you won't even get SIGCHLD signals (though waitpid() might still work).

I'm not sure if the child is guaranteed to survive, either, since a write to a pipe with no reader (namely, the redirected stdout of the child) should kill the child with a SIGPIPE.

If you want your parent process to pick up where the child left off, you need to spawn the child to write to a file, usually in /tmp or /var/log, and have it record its PID like you are now (the usual location is /var/run). (Having it write to a named pipe risks getting it killed with SIGPIPE as above.) If you suffix your filename with the PID, then it becomes easy for the manager process to figure out which file belongs to which daemon.