Human-readable binary data using Python

Question

My work requires that I perform a mathematical simulation whose parameters come from a binary file. The simulator can read such binary file without a problem.

However, I need to peek inside the binary file to make sure the parameters are what I need them to be, and I cannot seem to be able to do it.

I would like to write an script in Python which would allow me to read in the binary file, search for the parameters that I care about, and display what their values are.

What I know about the binary file:

It represents simple text (as opposed to an image or soud file). There is a piece of code that can "dump" the file into a readable format: if I open that dump in Emacs I will find things like:

CENTRAL_BODY = 'SUN'

All the file is just a series of similar instructions. I could use that dump code, but I much rather have Python do that.

This seems to be a very trivial question, and I apologize for not knowing better. I thought I was a proficient programmer!

Many thanks.

Answer 1

You can read the file's content into a string in memory:

thedata = open(thefilename, 'rb').read()

and then locate a string in it:

where = thedata.find('CENTRAL_BODY')

and finally slice off the part you care about:

thepart = thedata[where:where+50]  # or whatever length

and display it as you prefer (e.g. find the string value by locating within thepart an = sign, then the first following quote, then the next quote after that).

Answer 2

If it's a binary file, you will need to use the struct module. You will need to know how the data is formatted in the file. If that is not documented, you will have to reverse engineer it.

Do you have source code of the other dumping program? You may be able to just port that to Python

We can probably help you better if we can see what the binary file and the corresponding dump looks like

Answer 3

It sounds like this 'dump' program already does what you need: interpreting the binary file. I guess my approach would be to write a python program that can take a dump'ed file, extract the parameters you want and display them.

Then parse it with something like this:

myparms.py:

import sys

d = {}
for line in sys.stdin:
    parts = line.split("=",2)
    if len(parts) < 2:
        continue
    k = parts[0].strip()
    v = parts[1].strip()
    d[k] = v

print d['CENTRAL_BODY']

Use this like:

dump parameters.bin | python myparms.py

You didn't mention a platform or provide details about the dump'ed format, but this should be a place to start.

Answer 4

You have to know the format the data is stored in; there's simply no way around that.

If there's no written spec on it, try to open it in a hex editor and study the format, using the text-dump as a reference. If you can get the source code for the tool that creates the text-dumps, that would help you alot.

Keep in mind that the data could be scrambled in someway or another, e.g. rot13.