Python *.py, *.pyo, *.pyc: Which can be eliminated for an Embedded System?

Question

To squeeze into the limited amount of filesystem storage available in an embedded system I'm currently playing with, I would like to eliminate any files that could reasonably be removed without significantly impacting functionality or performance. The *.py, *.pyo, and *.pyc files in the Python library account for a sizable amount of space, I'm wondering which of these options would be most reasonable for a Python 2.6 installation in a small embedded system:

1. Keep *.py, eliminate *.pyc and *.pyo (Maintain ability to debug, performance suffers?)
2. Keep *.py and *.pyc, eliminate *.pyo (Does optimization really buy anything?)
3. Keep *.pyc, eliminate *.pyo and *.py (Will this work?)
4. Keep *.py, *.pyc, and *.pyo (All are needed?)

Answer 1

Number 3 should and will work. You do not need the .pyo or .py files in order to use the compiled python code.

Answer 2

http://www.network-theory.co.uk/docs/pytut/CompiledPythonfiles.html

When the Python interpreter is invoked with the -O flag, optimized code is generated and stored in ‘.pyo’ files. The optimizer currently doesn't help much; it only removes assert statements.

Passing two -O flags to the Python interpreter (-OO) will cause the bytecode compiler to perform optimizations that could in some rare cases result in malfunctioning programs. Currently only doc strings are removed from the bytecode, resulting in more compact ‘.pyo’ files.

My suggestion to you?

Use -OO to compile only .pyo files if you don't need assert statements and __doc__ strings.

Otherwise, go with .pyc only.

Edit

I noticed that you only mentioned the Python library. Much of the python library can be removed if you only need part of the functionality.

I also suggest that you take a look at tinypy which is large subset of Python in about 64kb.

Answer 3

What it ultimately boils down to is that you really only need one of the three options, but your best bet is to go with .pys and either .pyos or .pycs.

Here's how I see each of your options:

1. If you put the .pys in a zip file, you won't see pycs or pyos built. It should also be pointed out that the performance difference is only in startup time, and even then isn't too great in my experience (your milage may vary though). Also note that there is a way to prevent the interpreter from outputting .pycs as Algorias points out.
2. I think that this is an ideal option (either that or .pys and .pyos) because you get the best mix of performance, debuggability and reliability. You don't necessarily need a source file and compiled file though.
3. If you're really strapped for space and need performance, this will work. I'd advise you to keep the .pys if at all possible though. Compiled binaries (.pycs or .pyos) don't always transfer to different versions of python.
4. It's doubtful that you'll need all three unless you plan on running in optimized mode sometimes and non-optimized mode sometimes.

In terms of space it's been my (very anecdotal) experience that .py files compress the best compared to .pycs and .pyos if you put them in a zipfile. If you plan on compressing the files, .pyos don't tend to gain a lot in terms of sheer space because docstrings tend to compress fairly well and asserts just don't take up that much space.

Answer 4

I would recommend keeping only .py files. The difference in startup time isn't that great, and having the source around is a plus, as it will run under different python versions without any issues.

As of python 2.6, setting sys.dont_write_bytecode to True will suppress compilation of .pyc and .pyo files altogether, so you may want to use that option if you have 2.6 available.