twisted: check whether a deferred has already been called

Question

This is what I'm trying to accomplish. I'm making a remote call to a server for information, and I want to block to wait for the info. I created a function that returns a Deferred such that when the RPC comes in with the reply, the deferred is called. Then I have a function called from a thread that goes threads.blockingCallFromThread(reactor, deferredfunc, args).

If something goes wrong - for example, the server goes down - then the call will never un-block. I'd prefer the deferred to go off with an exception in these cases.

I partially succeeded. I have a deferred, onConnectionLost which goes off when the connection is lost. I modified my blocking call function to:

    deferred = deferredfunc(args)
    self.onConnectionLost.addCallback(lambda _: deferred.errback(
        failure.Failure(Exception("connection lost while getting run"))))
    result = threads.blockingCallFromThread(
        reactor, lambda _: deferred, None)
    return result

This works fine. If the server goes down, the connection is lost, and the errback is triggered. However, if the server does not go down and everything shuts down cleanly, onConnectionLost still gets fired, and the anonymous callback here attempts to trigger the errback, causing an AlreadyCalled exception to be raised.

Is there any neat way to check that a deferred has already been fired? I want to avoid wrapping it in a try/except block, but I can always resort to that if that's the only way.

Answer 1

There are ways, but you really shouldn't do it. Your code that is firing the Deferred should be keeping track of whether it's fired the Deferred or not in the associated state. Really, when you fire the Deferred, you should lose track of it so that it can get properly garbage collected; that way you never need to worry about calling it twice, since you won't have a reference to it any more.

Also, it looks like you're calling deferredfunc from the same thread that you're calling blockingCallFromThread. Don't do that; functions which return Deferreds are most likely calling reactor APIs, and those APIs are not thread safe. In fact, Deferred itself is not thread safe. This is why it's blockingCallFromThread, not blockOnThisDeferredFromThread. You should do blockingCallFromThread(reactor, deferredfunc, args).

If you really want errback-if-it's-been-called-otherwise-do-nothing behavior, you may want to cancel the Deferred.