views:

67

answers:

2

I've decided to dip my toe into the world of asynchronous python with the help of twisted. I've implemented some of the examples from the documentation, but I'm having a difficult time finding an example of the, very simple, client I'm trying to write.

In short I'd like a client which establishes a tcp connection with a server and then sends simple "\n" terminated string messages off of a queue object to the server. The server doesn't ever respond with any messages so my client is fully unidirectional. I /think/ that what I want is some combination of this example and the twisted.internet.protocols.basic.LineReceiver convenience protocol. This feels like it should be just about the simplest thing one could do in twisted, but none of the documentation or examples I've seen online seem to fit quite right.

A: 

I also had difficulties getting started with Twisted, and found on SO ( http://stackoverflow.com/questions/3374481/python-twisted-tutorial-question ) a link to this series of articles: http://krondo.com/blog/?page_id=1327

Very well done, clean, clear, simple.

Hope it helps.

Xavier
+1  A: 

What I have done is not used a Queue but I am illustrating the code that sends a line, once a connection is made. There are bunch of print stuff that will help you understand on what is going on.

Usual import stuff:

from twisted.web import proxy
from twisted.internet import reactor
from twisted.internet import protocol
from twisted.internet.protocol import ReconnectingClientFactory 
from twisted.protocols import basic
from twisted.python import log
import sys
log.startLogging(sys.stdout)

You create a protocol derived from line receiver, set the delimiter. In this case, I simply write a string "www" once the connection is made. The key thing is to look at protocol interface at twisted.internet.interface.py and understand the various methods of protocol and what they do and when they are called.

class MyProtocol(basic.LineReceiver):

    #def makeConnection(self, transport):
    #    print transport       

    def connectionLost(self, reason):
        print reason
        self.sendData = False

    def connectionMade(self):
        print "connection made"
        self.delimiter = "\n"
        self.sendData = True
        print self.transport
        self.sendFromQueue()

    def sendFromQueue(self):
        while self.sendData:
            msg = dataQueue.get()
            self.sendLine(msg)
            # you need to handle empty queue
            # Have another function to resume 

Finally, A protocol factory that will create a protocol instance for every connection. Look at method : buildProtcol.

class myProtocolFactory():
    protocol = MyProtocol

    def doStart(self):
        pass

    def startedConnecting(self, connectorInstance):
        print connectorInstance

    def buildProtocol(self, address):
        print address
        return self.protocol()

    def clientConnectionLost(self, connection, reason):
        print reason
        print connection

    def clientConnectionFailed(self, connection, reason):
        print connection
        print reason

    def doStop(self):
        pass

Now you use a connector to make a connection:

reactor.connectTCP('localhost', 50000, myProtocolFactory())
reactor.run()

I ran this and connected it to an server that simply prints what it receives and hence send no ack back. Here is the output:

1286906080.08   82     INFO 140735087148064 __main__ conn_made: client_address=127.0.0.1:50277
1286906080.08   83    DEBUG 140735087148064 __main__ created handler; waiting for loop
1286906080.08   83    DEBUG 140735087148064 __main__ handle_read
1286906080.08   83    DEBUG 140735087148064 __main__ after recv
'www\n'

Recieved: 4

The above example if not fault tolerant. To reconnect , when a connection is lost, you can derive your protocol factory from an existing twisted class - ReconnectingClientFactory. Twisted has almost all the tools that you would need :)

class myProtocolFactory(ReconnectingClientFactory):
    protocol = MyProtocol

    def buildProtocol(self, address):
        print address
        return self.protocol()

For further reference

I suggest that you read : http://krondo.com/?page_id=1327

[Edited: As per comment below]

pyfunc
`basic.LineReceiver.delimiter = "\n"`? Really? How awful. What do you have against `self.delimiter = "\n"`? It's shorter *and* less insane.
Jean-Paul Calderone
@Jean-Paul Calderone: +1 I am learning too. Thanks. I had written this in jiffy and hence the mistake
pyfunc
This is about how far I made it in my code, but what I don't understand is in the connectionMade method to the MyProtocol class you have a single function call. I want to have an async loop that pulls those values from a queue. If I'm understanding your code correctly in the event that the connection is made and immediately disconnects before I've finished sending all my queued messages then either the loop will block OR it will continue but will drop messages on the ground since the connection has gone away. Am I reading that right?
Dave Rawks
` def connectionMade(self):`` print "connection made"`` self.delimiter = "\n"`` print self.transport`` while True:`` msg = dataQueue.get()`` self.sendLine(msg)`
Dave Rawks
@Dave Rawks: I have not used the Queue at all in my example. What you can do is set a flag when the connection is made, write code to fetch from the queue and send it via sendLine. Use a flag to tell you if it is still connected. When the connectLost is called, toggle the flag.
pyfunc
@Dave Rawks: Made some changes that should help. When you use reconnecting factory, when a connection is lost, re-connection happens automatically. Use a flag to tell if you can send data over the connection. Reset this flag when the connection is lost.
pyfunc
I played with the above code a bunch and couldn't get it to work correctly. The loop inside the sendFromQueue method causes the connectionMade method to block which means the flag never gets reset when the connection is lost. I did figure out a solution though... twisted.internet.task.loopingcall can be used to call the function which handles popping off the queue and sending, then the connectionMade and connectionLost methods can be used to start and stop that looping call. Everything works awesome now.
Dave Rawks
@Dave Rawks : Even if you are using the Queue in while loop. It should have worked, if you use the yield to get elements of the queue and you are not blocked if the queue is empty. Basically, check if there is a blocking operation. This will work against Twisted even if you use loopingCall
pyfunc