Embed pickle (or arbitrary) data in python script

Question

In Perl, the interpreter kind of stops when it encounters a line with

__END__

in it. This is often used to embed arbitrary data at the end of a perl script. In this way the perl script can fetch and store data that it stores 'in itself', which allows for quite nice opportunities.

In my case I have a pickled object that I want to store somewhere. While I can use a file.pickle file just fine, I was looking for a more compact approach (to distribute the script more easily).

Is there a mechanism that allows for embedding arbitrary data inside a python script somehow?

Answer 1

In Python, you can use """ (triple-quoted strings) to embed long runs of text data in your program.

In your case, however, don't waste time on this.

If you have an object you've pickled, you'd be much, much happier dumping that object as Python source and simply including the source.

The repr function, applied to most objects, will emit a Python source-code version of the object. If you implement __repr__ for all of your custom classes, you can trivially dump your structure as Python source.

If, on the other hand, your pickled structure started out as Python code, just leave it as Python code.

Answer 2

With pickle you can also work directly on strings.

s = pickle.dumps(obj)
pickle.loads(s)

If you combine that with """ (triple-quoted strings) you can easily store any pickled data in your file.

Answer 3

If the data is not particularly large (many K) I would just .encode('base64') it and include that in a triple-quoted string, with .decode('base64') to get back the binary data, and a pickle.loads() call around it.