Putting a thread to sleep until event X occurs

Question

I'm writing to many files in a threaded app and I'm creating one handler per file. I have HandlerFactory class that manages the distribution of these handlers. What I'd like to do is that

thread A requests and gets foo.txt's file handle from the HandlerFactory class

thread B requests foo.txt's file handler

handler class recognizes that this file handle has been checked out

handler class puts thread A to sleep

thread B closes file handle using a wrapper method from HandlerFactory

HandlerFactory notifies sleeping threads

thread B wakes and successfully gets foo.txt's file handle

This is what I have so far,

 def get_handler(self, file_path, type):
  self.lock.acquire()
  if file_path not in self.handlers:
   self.handlers[file_path] = open(file_path, type)
  elif not self.handlers[file_path].closed:
   time.sleep(1)
  self.lock.release()
  return self.handlers[file_path][type]

I believe this covers the sleeping and handler retrieval successfully, but I am unsure how to wake up all threads, or even better wake up a specific thread.

Answer 1

Hi. You do realize that Python has a giant lock, so that most of the benefits of multi-threading you do not get, right?

Unless there is some reason for the master thread to do something with the results of each worker, you may wish to consider just forking off another process for each request. You won't have to deal with locking issues then. Have the children do what they need to do, then die. If they do need to communicate back, do it over a pipe, with XMLRPC, or through a sqlite database (which is threadsafe).

Answer 2

What you're looking for is known as a condition variable.

Condition Variables

Here is an article on the usage within Python.

Answer 3

Looks like you want a threading.Semaphore associated with each handler (other synchronization objects like Events and Conditions are also possible, but a Semaphore seems simplest for your needs). (Specifically, use a BoundedSemaphore: for your use case, that will raise an exception immediately for programming errors that erroneously release the semaphone more times than they acquire it -- and that's exactly the reason for being of the bounded version of semaphones;-).

Initialize each semaphore to a value of 1 when you build it (so that means the handler is available). Each using-thread calls acquire on the semaphore to get the handler (that may block it), and release on it when it's done with the handler (that will unblock exactly one of the waiting threads). That's simpler than the acquire/wait/notify/release lifecycle of a Condition, and more future-proof too, since as the docs for Condition say:

The current implementation wakes up exactly one thread, if any are waiting. However, it’s not safe to rely on this behavior. A future, optimized implementation may occasionally wake up more than one thread.

while with a Semaphore you're playing it safe (the semantics whereof are safe to rely on: if a semaphore is initialized to N, there are at all times between 0 and N-1 [[included]] threads that have successfully acquired the semaphore and not yet released it).