Reading a binary file in Python into a struct

Question

How do I go about opening a binary data file in Python and reading back the values one long at a time, into a struct. I have something like this at the moment but I think this will keep overwriting idList, I want to append to it, so I end up with a tuple of all the long values in the file -

    file = open(filename, "rb")

        try:
            bytes_read = file.read(struct.calcsize("=l"))
            while bytes_read:

                # Read 4 bytes(long integer)
                idList = struct.unpack("=l", bytes_read)

                bytes_read = file.read(struct.calcsize("=l"))
        finally:
            file.close()

Thanks.

Answer 1

Simplest (python 2.6 or better):

import array
idlist = array.array('l')
with open(filename, "rb") as f:
  while True:
    try: idlist.fromfile(f, 2000)
    except EOFError: break
idtuple = tuple(idlist)

Tuples are immutable, so they can't be built incrementally: so you have to build a different (mutable) sequence, then call tuple on it at the end. If you don't actually need specifically a tuple, of course, you can save the last, costly step and keep the array or list or whatever. Avoiding trampling over built-in names like file is advisable anyway;-).

If you have to use the struct module for a job that's best handled by the array module (e.g., because of a bet),

idList = [ ]
with open(filename, "rb") as f:
    while True:
        bytes_read = file.read(struct.calcsize("=l"))
        if not bytes_read: break
            oneid = struct.unpack("=l", bytes_read)[0]
            idList.append(oneid)

The with statement (also available in 2.5 with an import form the future) is better than the old try/finally in clarity and conciseness.

Answer 2

Change

idList = struct.unpack("=l", bytes_read)

to

idList.append(struct.unpack("=l", bytes_read)[0])