Writing binary data to a socket (or file) with Python

Question

Let's say I have a socket connection, and the 3rd party listener on the other side expects to see data flowing in a very structured manner. For example, it looks for an unsigned byte that denotes a type of message being sent, followed by an unsigned integer that denotes the length of message, then another unsigned byte which is really a bit field with some flags set or unset and etc...

How would I do this in python? I'm just wondering how to reliably generate this data and make sure I'm sending it correctly (i.e. that I'm really sending an unsigned byte rather than say a signed integer or worse, a string).

Answer 1

Use the struct module to build a buffer and write that.

Answer 2

At the lowest level, socket I/O consists of reading or writing a string of byte values to a socket. To do this, I encode the information to be written as a string of characters containing the byte values, and write it to the socket. I do this by creating a superstring, and then appending one character at a time. for example, to create a Modbus/Ethernet read request:

    readRequest = """"""
    readRequest += chr(self.transactionID / 0x100)  # Transaction ID MSB         (0)
    readRequest += chr(self.transactionID % 0x100)  # Transaction ID LSB         (1)
    readRequest += chr(0)                           # Protocol ID MSB (Always 0) (2)
    readRequest += chr(0)                           # Protocol ID LSB (Always 0) (3)
    readRequest += chr(0)                           # Length MSB (Always 0)      (4)
    readRequest += chr(6)                           # Length LSB (Always 6)      (5)
    readRequest += chr(0)                           # Unit ID (Always 0)         (6)

    readRequest += chr(0x04)                        # Function code 4     (0)
    readRequest += chr(startOffset / 0x100)         # Starting offset MSB (1)
    readRequest += chr(startOffset % 0x100)         # Starting offset LSB (2)
    readRequest += chr(0)                           # Word count MSB      (3)
    readRequest += chr(2 * nToRead)                 # Word count LSB      (4)

    sockOutfile.write(readRequest)

To convert multibyte values into character strings so they can be appended onto the I/O string, use the 'Pack()' function in the struct module. This function converts one or more single or multiple byte values into a string of individual byte values.

Of course, this method is about as simple as a hammer. It will need to be fixed when the default character encoding in a string is Unicode instead of ASCII.

Answer 3

A very elegant way to handle theses transitions between Python objects and a binary representation (both directions) is using the Construct library.

In their documentation you'll find many nice examples of using it. I've been using it myself for several years now for serial communications protocols and decoding binary data.