Python - multithreading / multiprocessing, very strange problem.

Question

Module run via python myscript.py (not shell input)

import uuid
import time
import multiprocessing


def sleep_then_write(content):
    time.sleep(5)
    print(content)

if __name__ == '__main__':
    for i in range(15):
        p = multiprocessing.Process(target=sleep_then_write,
                                    args=('Hello World',))
        p.start()

print('Ah, what a hard day of threading...')

This script output the following:

Ah, what a hard day of threading...
Ah, what a hard day of threading...
Ah, what a hard day of threading...
Ah, what a hard day of threading...
Ah, what a hard day of threading...
Ah, what a hard day of threading...
Ah, what a hard day of threading...
Ah, what a hard day of threading...
Ah, what a hard day of threading...
AAh, what a hard day of threading..
h, what a hard day of threading...
Ah, what a hard day of threading...
Ah, what a hard day of threading...
Ah, what a hard day of threading...
Ah, what a hard day of threading...
Ah, what a hard day of threading...
Hello World
Hello World
Hello World
Hello World
Hello World
Hello World
Hello World
Hello World
Hello World
Hello World
Hello World
Hello World
Hello World
Hello World
Hello World

Firstly, why the heck did it print the bottom statement sixteen times (one for each process) instead of just the one time?

Second, notice the AAh, and h, about half way down; that was the real output. This makes me wary of using threads ever, now.

(Windows XP, Python 2.6.4, Core 2 Duo)

Answer 1

Due to Windows's lack of the standard fork system call, the multiprocessing module works somewhat funny on Windows. For one, it imports the main module (your script) once for each process. For a detailed explanation, see the "Windows" subtitle at http://docs.python.org/library/multiprocessing.html#multiprocessing-programming

Answer 2

multiprocessing works by starting several processes. Each process loads a copy of your script (that way it has access to the "target" function), and then runs the target function.

You get the bottom print statement 16 times because the statement is sitting out there by itself and gets printed when you load the module. Put it inside a main block and it wont:

if __name__ == "__main__":
    print('Ah, what a hard day of threading...')

Regarding the "AAh" - you have multiple processes going and they'll produce output as they run, so you simply have the "A" from one process next to the "Ah" from another.

When dealing with multi process or multi threaded environments you have to think through locking and communication. This is not unique to multiprocessing; any concurrent library will have the same issues.