Spawning a thread in python

Question

I have a series of 'tasks' that I would like to run in separate threads. The tasks are to be performed by separate modules. Each containing the business logic for processing their tasks.

Given a tuple of tasks, I would like to be able to spawn a new thread for each module as follows.

from foobar import alice, bob charles
data = getWorkData()
# these are enums (which I just found Python doesn't support natively) :(
tasks = (alice, bob, charles)

for task in tasks
  # Ok, just found out Python doesn't have a switch - @#$%!
  # yet another thing I'll need help with then ...
  switch
    case alice:
      #spawn thread here - how ?
      alice.spawnWorker(data)

No prizes for guessing I am still thinking in C++. How can I write this in a Pythonic way using Pythonic 'enums' and 'switch'es, and be able to run a module in a new thread.

Obviously, the modules will all have a class that is derived from a ABC (abstract base class) called Plugin. The spawnWorker() method will be declared on the Plugin interface and defined in the classes implemented in the various modules.

Maybe, there is a better (i.e. Pythonic) way of doing all this?. I'd be interested in knowing

[Edit]

I've just been reading a bot more and it seems Python does not implement threading in the true sense (at least, not in the sense that a C++ programmer would think). In any case thats not a show stopper for me. Each of the tasks are fairly time consuming, and I dont want to hold up starting one task until another has completed, thats why I am using threading. Time slicing does not bother me much - so long as they are all started pretty much at the same time (or shortly after each other) Python can then timeslice between the treads as much as it wants - its fine by me.

I have seen an answer to a similar question here on SO.

A user provides a simple class for threading as follows:

import threading
class Foo (threading.Thread):
    def __init__(self,x):
        self.__x = x
        threading.Thread.__init__(self)
    def run (self):
          print str(self.__x)

for x in xrange(20):
    Foo(x).start()

I am thinking of using this for my ABC Plugin. My question then is where do I put the code where the actual task gets done (i.e. the business logic). I assume this goes in the run() method of the Foo class (obvious question I know, but I dont want to make any assumptions).

Is my thinking on the right track or flawed (if flawed - what have I missed?)

Answer 1

Python can hold functions as objects. To overcome the limitation on lacking a switch may I suggest the following:

case_alice = lambda data : alice.spawnWorker(data)

my_dict[alice] = case_alice

forming a dictionary to hold your "case" statements.

Let me take it even further:

data = getWorkData()
case_alice = lambda d : alice.spawnWorker( d )
case_bob = lambda d : bob.spawnWorker( d )
case_charles = lambda d : charles.spawnWorker( d )

switch = { alice : case_alice, bob : case_bob, charles : case_charles }
spawn = lambda person : switch[ person ]( data )
[ spawn( item ) for item in (alice, bob, charles )]

Answer 2

sequential execution

from foobar import alice, bob charles

for fct in (alice, bob charles):
    fct()

parallel execution

from threading import Thread
from foobar import alice, bob charles

for fct in (alice, bob charles):
    Thread(target=fct).start()

Answer 3

import threading
from foobar import alice, bob, charles

data = get_work_data() # names_in_pep8 are more Pythonic than camelCased

for mod in [alice, bob, charles]:
    # mod is an object that represent a module
    worker = getattr(mod, 'do_work')
    # worker now is a reference to the function like alice.do_work
    t = threading.Thread(target=worker, args=[data])
    # uncomment following line if you don't want to block the program
    # until thread finishes on termination
    #t.daemon = True 
    t.start()

Put your logic in do_work functions of corresponding modules.

Answer 4

Instead of switch-case, why not use a proper polymorphism? For example, here what you can do with duck typing in Python:

In, say, alice.py:

def do_stuff(data):
    print 'alice does stuff with %s' % data

In, say, bob.py:

def do_stuff(data):
    print 'bob does stuff with %s' % data

Then in your client code, say, main.py:

import threading
import alice, bob

def get_work_data():
    return 'data'

def main():
    tasks = [alice.do_stuff, bob.do_stuff]
    data = get_work_data()
    for task in tasks:
        t = threading.Thread(target=task, args=(data,))
        t.start()

Let me know if I need to clarify.