Is it possible to run function in a subprocess without threading or writing a separate file/script.

Question

import subprocess

def my_function(x):
    return x + 100

output = subprocess.Popen(my_function, 1) #I would like to pass the function object and its arguments
print output 
#desired output: 101

I have only found documentation on opening subprocesses using separate scripts. Does anyone know how to pass function objects or even an easy way to pass function code?

Answer 1

I think you're looking for something more like the multiprocessing module:

http://docs.python.org/library/multiprocessing.html#the-process-class

The subprocess module is for spawning processes and doing things with their input/output - not for running functions.

Here is a multiprocessing version of your code:

from multiprocessing import Process, Queue

def my_function(q, x):
    q.put(x + 100)

if __name__ == '__main__':
    queue = Queue()
    p = Process(target=my_function, args=(queue, 1))
    p.start()
    p.join()                    # this deadlocks
    result = queue.get()
    print result

Answer 2

You can use the standard Unix fork system call, as os.fork(). fork() will create a new process, with the same script running. In the new process, it will return 0, while in the old process it will return the process ID of the new process.

child_pid = os.fork()
if child_pid == 0:
  print "New proc"
else:
  print "Old proc"

For a higher level library, that provides multiprocessing support that provides a portable abstraction for using multiple processes, there's the multiprocessing module. There's an article on IBM DeveloperWorks, Multiprocessing with Python, with a brief introduction to both techniques.

Answer 3

I believe that you are looking for the multiprocessing module.