Threads in Python

Question

General tutorial or good resource on how to use threads in Python?

When to use threads, how they are effective, and some general background on threads [specific to Python]?

Answer 1

Threads should be used when you want two things to run at once, or want something to run in the background without slowing down the main process.
My recommendation is to only use threads if you have to. They generally add complexity to a program.
The main documentation for threading is here: http://docs.python.org/library/threading.html
Some examples are here:
http://www.devshed.com/c/a/Python/Basic-Threading-in-Python/1/
http://linuxgazette.net/107/pai.html
http://www.wellho.net/solutions/python-python-threads-a-first-example.html

Answer 2

There are several tutorials here.

Answer 3

There is a fantastic pdf, Tutorial on Threads Programming with Python by Norman Matloff and Francis Hsu, of University of California, Davis.

Threads should be avoided whenever possible. They add much in complexity, synchronization issues and hard to debug issues. However some problems require them (i.e. GUI programming), but I would encourage you to look for a single-threaded solution if you can.

Answer 4

One thing to remember before spending time and effort in writing a multi-threaded Python application is that there is a Global Interpreter Lock (GIL), so you won't actually be running more than one thread at a time.

This makes threading unsuitable for trying to take advantage of multiple cores or CPUs. You may get some speedup from multiplexing other resources (network, disk, ...), but it's never been particularly noticeable in my experience.

In general, I only use threads when there are several logically separate tasks happening at once, and yet I want them all in the same VM. A thread pulling data from the web and putting it on a Queue, while another thread pops from the Queue and writes to a database, something like that.

With Python 2.6, there is the new multiprocessing module which is pretty cool - it's got a very similar interface to the threading module, but actually spawns new OS processes, sidestepping the GIL.