I'm using the Twisted framework, and am getting RPCs asynchronously. I have another function which does a task every 2 seconds, and sleeps in between. This is called through reactor.callInThread. These depend on a shared resources, so I need some thread-safe way of accessing them. How does one go about using critical sections / mutexes / locks in twisted?
+2
A:
Though you can use threads in twisted, the usual idiom with twisted is to do RPC asyncronously using a single thread. Thats one of its advantages. The twisted framework will run the reactor and call your handler events when RPC results are ready for you. Then your code runs, and when your handler exits, control goes back to the reactor which will call the next handler that has code ready. So even though a lot of things are going on in paralell, twisted ensures that only one of your functions is running at a time, so you shouldn't need any mutexing, just maintaining state variables so your callbacks know what the current context they are operating in is enough.
If you are explictly create threads and use them with the twisted framework running you'd probably need something like Standard Python Mutex, though you'd need to be very careful not to ever have your main Reactor callback thread waiting on a mutex for any length of time as callbacks inside the reactor aren't supposed to block.
bdk
2009-06-26 23:18:48
hmm ok. Well, to be more specific, I have one function call running as a result of "reactor.callInThread". This function does something every 2 seconds. How does the reactor handle that? i want to make sure the RPC doesn't run at the same time that that function is doing something.
2009-06-26 23:21:48
hah - is the answer to use a task.LoopingCall instead?
2009-06-26 23:22:23
If you aren't using the multithreaded approach for performance (CPU scaling,etc) reasons, and the function in the other thread doesn't run for too long without exiting (as to block the reactor thread), that would seem to be acceptable. Honestly if you need the threaded approach I'm not positive what the right way to do the mutexing is since the reactor thread isn't allowed to block.
bdk
2009-06-26 23:47:36
A:
Twisted lets you write event-driven code in a single thread. Multiple events can write to standard Python non-thread-safe data structures in a safe matter, and non-thread-safe data structures can be used as mutexes. If you do start using threads, then you have to worry about these things. But you don't have to use them.
So, as commented: use task.LoopingCall or reactor.CallLater for your task. Never call time.sleep(), let the reactor call your task at the right time (and do other work in between). Respond to your RPCs as they come.
There won't be two threads running your code at once. However, you don't know the order in which your callbacks will be called. Once you relinquish control to a Deferred, application state may have changed by the time you get it back.
Karl Anderson
2009-08-19 22:29:13