Synchronising multiple threads in python

Question

I have a problem where I need x threads to wait until they have all reached a synchronization point. My solution uses the synchronise method below which is called by each threaded function when they need to synchronise.

Is there a better way to do this?

thread_count = 0
semaphore = threading.Semaphore()
event = threading.Event()

def synchronise(count):
    """ All calls to this method will block until the last (count) call is made """
    with semaphore:
        thread_count += 1
        if thread_count == count:
            event.set()

    event.wait()

def threaded_function():
    # Do something

    # Block until 4 threads have reached this point
    synchronise(4)

    # Continue doing something else

Answer 1

The functionality you want is called a "barrier". (Unfortunately that term has 2 meanings when talking about threading. So if you Google it, just ignore articles that talk about "memory barriers" - that's a very different thing).

Your code looks quite reasonable - it's simple and safe.

I couldn't find any "standard" implementations of barriers for Python, so I suggest you keep using your code.

Answer 2

There are many ways to synchronize threads. Many.

In addition to synchronize, you can do things like the following.

1. Break your tasks into two steps around the synchronization point. Start threads doing the pre-sync step. Then use "join" to wait until all threads finish step 1. Start new threads doing the post-sync step. I prefer this, over synchronize.

2. Create a queue; acquire a synchronization lock. Start all threads. Each thread puts an entry in the queue and waits on the synchronization lock. The "main" thread sits in a loop dequeueing items from the queue. When all threads have put an item in the queue, the "main" thread releases the synchronization lock. All other threads are now free to run again.

There are a number of interprocess communication (IPC) techniques -- all of which can be used for thread synchronization.