views:

151

answers:

1

I'm working on writing a Python client for Direct Connect P2P networks. Essentially, it works by connecting to a central server, and responding to other users who are searching for files.

Occasionally, another client will ask us to connect to them, and they might begin downloading a file from us. This is a direct connection to the other client, and doesn't go through the central server.

What is the best way to handle these connections to other clients? I'm currently using one Twisted reactor to connect to the server, but is it better have multiple reactors, one per client, with each one running in a different thread? Or would it be better to have a completely separate Python script that performs the connection to the client?

If there's some other solution that I don't know about, I'd love to hear it. I'm new to programming with Twisted, so I'm open to suggestions and other resources.

Thanks!

+3  A: 

Without knowing all the details of the protocol, I would still recommend using a single reactor -- a reactor scales quite well (especially advanced ones such as PollReactor) and this way you will avoid the overhead connected with threads (that's how Twisted and other async systems get their fundamental performance boost, after all -- by avoiding such overhead). In practice, threads in Twisted are useful mainly when you need to interface to a library whose functions could block on you.

Alex Martelli
This is very useful, thanks!However, how would a single reactor handle slow network writing? To transfer a file, even in very small chunks like 4 MB, it takes time to transmit this data to the other client. Is there some way to make the transport.write() method not block the rest of the program?
Alex Kloss
A proper `transport.write` in Twisted will not block: it will write what it can and keep the rest for the next time some more data can be written! (`writeSequence` is also often useful, but also non-blocking).
Alex Martelli
E.g., see the source of twisted.internet.abstract.FileDescriptor.write, at http://twistedmatrix.com/trac/browser/trunk/twisted/internet/abstract.py -- many transport classes inherit it and their `write` methods delegate eventually to that -- each class then implements `writeSomeData` to write some bytes without blocking, when feasible (it's called from `doWrite`), and return the number of bytes written.
Alex Martelli
Great, thanks for the help!
Alex Kloss