views:

43

answers:

2

Does python threading expose issues of memory visibility and statement reordering as Java does? Since I can't find any reference to a "Python Memory Model" or anything like that, despite the fact that lots of people are writing multithreaded Python code, I'm guessing that these gotchas don't exist here. No volatile keyword, for instance. But it doesn't seem to be stated explicitly anywhere that, for instance, a change in a variable in one thread is immediately visible to all other threads.

Maybe this stuff is all very obvious to Python programmers, but as a fearful Java programmer, I require a little extra reassurance :)

+1  A: 

Google: Global Interpreter Lock

http://docs.python.org/c-api/init.html#thread-state-and-the-global-interpreter-lock

Rawheiser
What does this have to do with the question?
Aaron Gallagher
A lot! Alex gets the check, for sure, but this answer ultimately led me to David Beazley's talk (http://blip.tv/file/2232410). Thanks @Rawheiser
philo
+5  A: 

There is no formal model for Python's threading (hey, after all, there wasn't one for Java's for years... hopefully, one will also eventually be written for Python).

In practice, no Python implementation performs any advanced optimization such as statement reordering or temporarily treating shared variables as thread-local ones -- and you can count on these semantics constraints even though they are not formally assured.

CPython in particular, as @Rawheiser's mention, uses a global interpreter lock; other implementations (PyPy, IronPython, Jython, ...) do not (so they can use multiple cores effectively with a threading model, while CPython requires multi-processing for the same purpose), so you should not count on that if you want to write code that's portable throughout Python implementations. (So, you shouldn't count on the "atomicity" of operations that only happen to be atomic in CPython because of the GIL, such as dictionary accesses -- in other Python implementations, multiple threads might be modifying a dict at once and cause errors unless you protect the dict with a lock or the like).

Alex Martelli
Even in CPython, dictionary access isn't atomic in all circumstances. If the hash/compare functions of the keys are written in Python, the GIL will be temporarily released between opcodes when those functions are executing. (You probably already know this, but I figure it's worth point out for other readers)
Daniel Stutzbach
Thank you! I've spent my entire morning learning about the GIL. Obviously (C)Python programming is going to involve a different way of looking at threads than I'm used to.@Daniel Stutzbach: I'm new to Python and would have overlooked that fact. Thank you.
philo
@philo, summarizing the multitasking situation for CPython: threads are going to help you only if you have I/O waits (that you can delegate to a thread) or heavy operations performed in a thread-safe Python extension (such as `numpy`). If your purpose is to use multiple cores for Python-coded, CPU-bound operations, use `multiprocessing` instead of `threading`.
Alex Martelli
@Alex Martelli, thanks for the excellent summary.
philo
@philo, you're welcome!
Alex Martelli