views:

95

answers:

2

I'm trying to figure out the differences between a task.LoopingCall and a reactor.callInThread in Twisted.

All my self.sendLine's in the LoopingCall are performed immediately. The ones in the callInThread are not. They're only sent after the one in the LoopingCall has finished. Even though I'm sending the right delimiter.

Why is that? What's the difference? Aren't they both threads?

This is the server:


from twisted.internet import reactor, protocol, task
from twisted.protocols import basic
from twisted.python import log
import sys
import time
import threading
import Queue

class ServerProtocol(basic.LineOnlyReceiver):
    delimiter = '\0'
    clientReady = 1

    def __init__(self):
        print 'New client has logged on. Waiting for initialization'

    def lineReceived(self, line):
        if line.startswith('I'):
            print 'Data started with I: '+line
            user = dict(uid=line[1:6], x=line[6:9], y=line[9:12])
            self.factory.users[user['uid']] = user
            log.msg(repr(self.factory.users))
            self.startUpdateClient(user)
            reactor.callInThread(self.transferToClient)
            self.sendLine(user['uid'] + ' - Beginning - Initialized')
            print user['uid'] + ' - Beginning - Initialized'
        elif line.startswith('P'):
            print 'Ping!'
        elif line[0:3] == 'ACK':
            print 'Received ACK'
            self.clientReady = 1
        #else:
            #self.transport.loseConnection()

    def _updateClient(self, user):
        if self._running == 0:
            self._looper.stop()
            return
        self._running -= 1
        self._test += 1
        print user['uid'] + ' Sending test data' + str(self._test)
        self.sendLine(user['uid'] + ' Test Queue Data #%d' % (self._test,) + '\0')

    def startUpdateClient(self, user):
        self._running, self._test = 25, 0
        self._looper = task.LoopingCall(self._updateClient, user)
        self._looper.start(1, now=False)
        print user['uid'] + ' - Startupdateclient'

    def transferToClient(self):
        test = 20
        while test > 0:
            if self.clientReady == 1:
                test = test-1
                print 'Reactor test ' + str(test) + ' - ' + str(time.time())
                self.clientReady = 0
                self.sendLine('This is reactortest ' + str(test) + ' - ' + str(time.time()) +' \0')

class Server(protocol.ServerFactory):
    protocol = ServerProtocol
    def __init__(self):
        self.users = {}

if __name__ == '__main__':
    log.startLogging(sys.stderr)
    reactor.listenTCP(2000, Server())
    reactor.run()

This is the client:


#!/usr/bin/env python

import socket
import time

host = 'localhost'
port = 2000
size = 1024
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect((host,port))
s.send('I12345070060\0')
running = 1

while running:
    s.send('ACK\0')
    data = s.recv(size)
    if data:
        print 'Received:', data 
    else:
        print 'Closing'
        s.close()
        running=0

+1  A: 

Have you looked at the docs for LoopingCall? No thread involved -- it runs (every second, the way you're calling its start method) on the main thread, i.e., typically, the thread of the reactor. callInThread is the only one of the two that causes the function to be run on a separate thread.

Alex Martelli
Ow, I was always looking at the Core documentation, not the API reference. But why doesn't the thread's sendLine work the same way as the one in the loop?
skerit
The bug in your code is that you're calling, directly from a non-main thread, callables that aren't meant to be called except from the main thread -- use `callFromThread` instead whenever you need to call twisted API entries from a non-main thread!
Alex Martelli
+2  A: 

Why is that? What's the difference? Aren't they both threads?

No. LoopingCall uses callLater; it runs the calls in the reactor.

All my self.sendLine's in the LoopingCall are performed immediately.

Yep, as they should be.

The ones in the callInThread are not.

It's not so much that they're not performed, it's that because you called a reactor API from a thread, which you are never ever ever allowed to do, you have put your program into a state where everything is completely broken, forever. Every future API call may produce bizarre, broken results, or no results, or random, inexplicable crashes.

You know, the normal way multithreaded programs work ;-).

To repeat: every API in twisted, with the sole exception of callFromThread (and by extension things which call callFromThread, like blockingCallFromThread), is not thread safe. Unfortunately, putting in warnings for every single API would be both a code maintenance nightmare, so several users have discovered this constraint in the same way that you have, by calling an API and noticing something weird.

If you have some code which runs in a thread that needs to call a reactor API, use callFromThread or blockingCallFromThread and it will dispatch the call to the reactor thread, where everything should work smoothly. However, for stuff like timed calls, there's really no need to use threads at all, and they would needlessly complicate your program.

Glyph
It's my very first time in Python. And my very first time ever using threading, can you tell? ;)I'll rethink my strategy!
skerit