Python - Launch a Long Running Process from a Web App

Question

I have a python web application that needs to launch a long running process. The catch is I don't want it to wait around for the process to finish. Just launch and finish.

I'm running on windows XP, and the web app is running under IIS (if that matters).

So far I tried popen but that didn't seem to work. It waited until the child process finished.

Answer 1

subprocess.Popen does that.

Answer 2

Instead of directly starting processes from your webapp, you could write jobs into a message queue. A separate service reads from the message queue and runs the jobs. Have a look at Celery, a Distributed Task Queue written in Python.

Answer 3

This almost works (from here):

from subprocess import Popen

pid = Popen(["C:\python24\python.exe", "long_run.py"]).pid
print pid
print 'done'

'done' will get printed right away. The problem is that the process above keeps running until long_run.py returns and if I close the process it kills long_run.py's process.

Surely there is some way to make a process completely independent of the parent process.

Answer 4

Ok, I finally figured this out! This seems to work:

from subprocess import Popen
from win32process import DETACHED_PROCESS

pid = Popen(["C:\python24\python.exe", "long_run.py"],creationflags=DETACHED_PROCESS,shell=True).pid
print pid
print 'done' 
#I can now close the console or anything I want and long_run.py continues!

Note: I added shell=True. Otherwise calling print in the child process gave me the error "IOError: [Errno 9] Bad file descriptor"