Testing twisted application - Load client

Question

Hi,

I've written a Twisted based server and I'd like to test it using twisted as well.

But I'd like to write a load test starting a bunch of request at the same time.

But I believe that I didn't get the concepts of Twisted, mainly client side, because I'm stucked with this problem:

    from twisted.internet import reactor, protocol
from threading import Thread
from twisted.protocols.basic import LineReceiver

__author__="smota"
__date__ ="$30/10/2009 17:17:50$"

class SquitterClient(LineReceiver):

    def connectionMade(self):
        self.sendLine("message from " % threading.current_thread().name);
        pass

    def connectionLost(self, reason):
        print "connection lost"

    def sendMessage(self, msg):
        for m in [ "a", "b", "c", "d", "e"]:
            self.sendLine(msg % " - " % m);

class SquitterClientFactory(protocol.ClientFactory):
    protocol = SquitterClient

    def clientConnectionFailed(self, connector, reason):
        print "Connection failed - goodbye!"
        reactor.stop()

    def clientConnectionLost(self, connector, reason):
        print "Connection lost - goodbye!"
        reactor.stop()

def createAndRun():
    f = SquitterClientFactory()
    reactor.connectTCP("localhost", 4010, f)
    reactor.run(installSignalHandlers=0)

# this connects the protocol to a server runing on port 8000
def main():
    for n in range(0,10):
        th=Thread(target=createAndRun)
        th.start()

# this only runs if the module was *not* imported
if __name__ == '__main__':
    main()

socket_client.py:35: DeprecationWarning: Reactor already running! This behavior is deprecated since Twisted 8.0
reactor.run(installSignalHandlers=0)

What am I missing?

How to test it?

Thank you,

Samuel

Answer 1

The direct cause for your failure is that you attemp to call run() on the reactor multiple times. You are supposed to ever only call run() once. I think you are expecting to have multiple reactors, each in its own thread, but actually you only have one. The bad thing is that having multiple reactors is difficult or impossible - the good thing is that it's also unnecessary. In fact you don't even need multiple threads. You can multiplex multiple client connections in one reactor almost as easily as you can listen for multiple connections.

Modifying your sample code, something like the following should work. The key idea is that you don't need multiple reactors to do things concurrently. The only thing that could ever be concurrent with the regular Python implementation is I/O anyway.

from twisted.internet import reactor, protocol
from twisted.protocols.basic import LineReceiver

__author__="smota"
__date__ ="$30/10/2009 17:17:50$"

class SquitterClient(LineReceiver):
    def connectionMade(self):
        self.messageCount = 0
        # The factory provides a reference to itself, we'll use it to enumerate the clients
        self.factory.n += 1
        self.name = "Client %d" %self.factory.n

        # Send initial message, and more messages a bit later
        self.sendLine("Client %s starting!" % self.name);
        reactor.callLater(0.5, self.sendMessage, "Message %d" %self.messageCount)

    def connectionLost(self, reason):
        print "connection lost"

    def sendMessage(self, msg):
        for m in [ "a", "b", "c", "d", "e"]:
            self.sendLine("Copy %s of message %s from client %s!" % (m, msg, self.name))
        if self.factory.stop:
            self.sendLine("Client %s disconnecting!" % self.name)
            self.transport.loseConnection()
        else:
            self.messageCount += 1
            reactor.callLater(0.5, self.sendMessage, "Message %d" %self.messageCount)

class SquitterClientFactory(protocol.ClientFactory):
    protocol = SquitterClient

    def __init__(self):
        self.n = 0
        self.stop = False

    def stopTest():
        self.stop = True

    def clientConnectionFailed(self, connector, reason):
        print "Connection failed - goodbye!"

    def clientConnectionLost(self, connector, reason):
        print "Connection lost - goodbye!"

# this connects the protocol to a server running on port 8000
def main():
    # Create 10 clients

    f = SquitterClientFactory()
    for i in range(10):
        reactor.connectTCP("localhost", 8000, f)

    # Schedule end of test in 10 seconds
    reactor.callLater(10, f.stopTest)

    # And let loose the dogs of war
    reactor.run()

# this only runs if the module was *not* imported
if __name__ == '__main__':
    main()