sending data between server and client in twisted

Question

I'm trying to transport data between server and client implemented with twisted. As far as I know, using self.transport.write([data]) will work only if data is a string. Is there any other way I can send an object of other type? Thank you!

Answer 1

Sockets carry bytes. That's the only kind of thing they carry. Any two endpoints of a TCP connection can only convey bytes to each other.

Bytes aren't the most useful data structure for every form of communication. So on top of this byte transport, we invent schemes for formatting and interpreting bytes. These are protocols.

Twisted represents protocols as classes, almsot always subclasses of twisted.internet.protocol.Protocol, which implement a particular scheme.

These classes have methods for turning something which isn't pure bytes into something which is pure bytes. For example, twisted.protocols.basic.NetstringReceiver is an implementation of the netstring protocol. It turns a particular number of bytes into bytes which represent both the number of bytes and the bytes themselves. This is a rather subtle protocol, since it's not instantly obvious that the byte count is information that needs to be conveyed as well.

These classes also interpret bytes received from the network, in their dataReceived method, according to the protocol they implement, and turn the resulting information into something more structured. NetstringReceiver uses the length information to accept exactly the right number of bytes from the network and then deliver them to its stringReceived callback as a single Python str instance.

Other protocols do more than NetstringReceiver. For example, twisted.protocols.ftp includes an implementation of the FTP protocol. FTP is a protocol geared towards passing file listings and files over a socket (or several sockets, actually). twisted.mail.pop3 implements POP3, a protocol for transferring email over sockets.

There are lots and lots of different protocols because there are lots and lots of different things you might want to do. Depending on exactly what you're trying to do, there are probably different ways to convert to and from bytes to make things easier or faster or more robust (and so on). So there's no single protocol that is ideal for the general case. That includes the case of "sending an object", since objects can take many different forms, and there may be many different reasons you want to send them, and many different ways you might want to handle things like mutation of an object you'd previously sent, and so on.

You probably want to spend a little time thinking about what kind of communication you need. This should suggest certain things about the protocol you'll select to do the communication.

For example, if you want to be able to call methods on Python objects that exist on the other side of a connection, then Twisted Spread might be interesting.

If you want something cross-language instead, and only need to convey simple types, like integers, strings, and lists, then XML-RPC (Twisted How-To) might be a better fit.

If you need a protocol that's more space efficient than XML-RPC and supports serialization of more complicated types, then AMP might be more appropriate.

And the list goes on. :)