convert a binary file in a list (python)

Question

Hi all,

I'd like to be able to open a binary file, and make a list (kind of array) with all the chars in, like : "\x21\x23\x22\x21\x22\x31" to ["\x21","\x23","\x22","\x21","\x22","\x31"] What would be the best solution to convert it ?

Thanks !

Answer 1

Assume that myfile.txt has 'abcdef\n' in it...

>>> fh = open('myfile.txt', 'rb')
>>> list(fh.read())
['a', 'b', 'c', 'd', 'e', 'f', '\n']

Answer 2

You can read the binary data into a string just like you would do with text data, just make sure to open the file in binary mode (the b flag in the call to open()):

with open('file.bin', 'rb') as f:
   data = f.read()

data now contains the characters from the file as a string, like "\x21\x23\x22\x21\x22\x31".

Answer 3

You need to understand that "\x21" and "!" are two ways of representing the same thing

so "\x21\x23\x22\x21\x22\x31" is the same as '!#"!"1'

>>> "\x21\x23\x22\x21\x22\x31"  == '!#"!"1'
True

>>> infile = open('infile.txt', 'rb')
>>> list(infile.read())
['!', '#', '"', '!', '"', '1']
>>> ['!', '#', '"', '!', '"', '1'] == ["\x21","\x23","\x22","\x21","\x22","\x31"]
True

So you see they are the same thing, but python always tries to pick the most user friendly way to display the characters

Answer 4

To make "a kind of array" of characters, an extremely efficient way (more than using a list!) is to use Python's standard library array module:

res = array.array('c')
with open('binaryfile', 'rb') as f:
    while True:
        try: res.fromfile(f, 1024 * 1024)
        except EOFError: break

This reads no more than a megabyte at a time (that's the 1024 * 1024), but keeps going until the file's all done -- you can tweak that behavior as you prefer, of course.