Connecting twice with twisted - how to do that correctly?

Question

Hi, I want to use twisted (and StarPy which is a protocol implementation for asterisk ami) to connect to an asterisk server. The application initiates a outgoing fax there. I found some hints on my problem, but I cannot find out how to handle this correctly.

The first fax is sent out correctly.

Problem is, if I call twisted for the second time, the application keeps hanging in main loop.

I know I may NOT do this like here:

from starpy import manager
from twisted.internet import reactor

def main():
    f = manager.AMIFactory(cUser, cPass)
    print "Login"
    df = f.login(cServer, cPort)

    def onLogin(protocol):
        print "Logoff again"
        df = protocol.logoff()

        def onLogoff( result ):
            print "Logoff erfolgt"
            reactor.stop()

        return df.addCallbacks( onLogoff, onLogoff )

    def onFailure( reason ):
        print "Login failed"
        print reason.getTraceback()

    df.addCallbacks( onLogin, onFailure )
    return df

if __name__ == "__main__":
    reactor.callWhenRunning( main )
    reactor.run(installSignalHandlers=0)
    print "runned the first time"

    reactor.callWhenRunning( main )
    reactor.run(installSignalHandlers=0)
    print "will never reach this point"

I simplified the code - it does nothing than login + logoff again. It will never return from the second reactor.run() call.

How is this done correctly? I'm stuck here - thanks in advance.

Best Regards, Florian.

Answer 1

You can't restart the reactor. In other words, you can call reactor.run() only once.

Instead can do everything you need in one reactor run.

Answer 2

As iny said, you need to do everything with just one call to reactor.run and reactor.stop.

If we consider the example code you posted, we see that it takes these steps:

1. Start the reactor
2. Connect, send a fax, disconnect
3. Stop the reactor
4. Start the reactor
5. Connect, send a fax, disconnect
6. Stop the reactor

If we only delete steps 3 and 4, then the program will actually be doing a pretty reasonable thing.

Here's how you implemented step 3:

def onLogoff( result ):
    print "Logoff erfolgt"
    reactor.stop()

This caused the first call to reactor.run to return, clearing the way for your implementation of step 4:

reactor.callWhenRunning( main )
reactor.run(installSignalHandlers=0)

So, the general idea here is going to be to jump right to step 5 instead of doing step 3 and 4. Consider what might happen if you redefine onLogoff like this:

def onLogoff( result ):
    print "Logoff erfolgt"
    main()

and deleting the last three lines of your example. This will actually give you an infinite loop, since the same onLogoff runs after the 2nd disconnect and starts a 3rd connection. However, you might remedy this with a parameter to the main function to control the restart behavior.

Once this makes sense, you may want to think about moving the retry logout out of the main function and into a callback defined in the __main__ block. This is a big part of the power of Deferreds: it lets you keep proper separation between the implementation of an event source (in this case, your fax sending function) and the code for dealing with the resulting events (sending a second fax, or exiting, in this case).

Answer 3

Thanks for your answers, I haven't implemented a solution right now but I know how I could do that now...

Here a short summary of things I learned.

First, in short - the problems I had with twisted:

1. I didn't understand the asynchronous basics of twisted. I used something like that in gui-frameworks but didn't see the benefit for a long time.
2. Second, I tried to think of an synchronous call of the event loop multiple times. This was necessary in my mind because I may not use more than one outgoing fax line at a time. Because the event loop of twisted is not restartable this is no option. As I could read in the docs "deferToThread" could help me here, but I think it's not the best solution.

In my concept I solved these problems:

• Use of defer.DeferredSemaphore() which allows me to limit concurrency to 1. I found an example here: http://oubiwann.blogspot.com/2008/06/async-batching-with-twisted-walkthrough.html
• calling twisted anynchronously and let the Defereds do the work (send a fax)
• Writing the fax status to the database and sending the receipt mail is called from the defereds

I needed a lot of re-thinking but as soon as you get it it looks really easy.

Thanks to iny and Jean-Paul Calderone for your help.

Answer 4

If you're still looking for a solution... I had this same issue. I have a script that uses Twisted to execute a program on a remote server. I needed a way to run that script synchronously from within a django application. What I ended up doing was making my Twisted script call the remote server and just print to stdout. Then from within my Django app I executed that script via subprocess.Popen and set stdout=PIPE so I could capture the output from my Twisted script and use it in my Django app.

This isn't really ideal, and pretty much defeats the purpose of Twisted, but this gets past the "not being able to call reactor.run() a second time, since the Twisted script runs in it's own process each time.

This did end up working great for me, and it sounds very similar to the same situation that you're in. I hope this helps. Good luck. (I can post some code samples if you think it would help, just let me know).