Is it feasible to compile Python to machine code?

Question

How feasible would it be to compile Python (possibly via an intermediate C representation) into machine code?

Presumably it would need to link to a Python runtime library, and any parts of the Python standard library which were Python themselves would need to be compiled (and linked in) too.

Also, you would need to bundle the Python interpreter if you wanted to do dynamic evaluation of expressions, but perhaps a subset of Python that didn't allow this would still be useful.

Would it provide any speed and/or memory usage advantages? Presumably the startup time of the Python interpreter would be eliminated (although shared libraries would still need loading at startup).

Answer 1

This might seem reasonable at first glance, however there are a lot of ordinary things in Python that aren't directly mappable to to a C representation without carrying over a lot of the Python runtime support. For example, duck typing comes to mind. Many functions in Python that read input can take a file or file-like object, as long as it supports certain operations, eg. read() or readline(). If you think about what it would take to map this type of support to C, you begin to imagine exactly the sorts of things that the Python runtime system already does.

There are utilities such as py2exe that will bundle a Python program and runtime into a single executable (as far as possible).

Answer 2

Psyco is a kind of just-in-time (JIT) compiler: dynamic compiler for Python, runs code 2-100 times faster, but it needs much memory.

In short: it run your existing Python software much faster, with no change in your source but it doesn't compile to object code the same way a C compiler would.

Answer 3

Try ShedSkin Python-to-C++ compiler, but it is far from perfect. Also there is Psyco - Python JIT if only speedup is needed. But IMHO this is not worth the effort. For speed-critical parts of code best solution would be to write them as C/C++ extensions.

Answer 4

Jython has a compiler targeting JVM bytecode. The bytecode is fully dynamic, just like the Python language itself! Very cool. (Yes, as Greg Hewgill's answer alludes, the bytecode does use the Jython runtime, and so the Jython jar file must be distributed with your app.)

Answer 5

PyPy is a project to reimplement Python in Python, using compilation to native code as one of the implementation strategies (others being a VM with JIT, using JVM, etc.). Their compiled C versions run slower than CPython on average but much faster for some programs.

Shedskin is an experimental Python-to-C++ compiler.

Pyrex is a language specially designed for writing Python extension modules. It's designed to bridge the gap between the nice, high-level, easy-to-use world of Python and the messy, low-level world of C.

Answer 6

As @Greg Hewgill says it, there are good reasons why this is not always possible. However, certain kinds of code (like very algorithmic code) can be turned into "real" machine code.

There are several options:

• Use Psyco, which emits machine code dynamically. You should choose carefully which methods/functions to convert, though.
• Use Cython, which is a Python-like language that is compiled into a Python C extension
• Use PyPy, which has a translator from RPython (a restricted subset of Python that does not support some of the most "dynamic" features of Python) to C or LLVM.
  • PyPy is still highly experimental
  • not all extensions will be present

After that, you can use one of the existing packages (freeze, Py2exe, PyInstaller) to put everything into one binary.

All in all: there is no general answer for your question. If you have Python code that is performance-critical, try to use as much builtin functionality as possible (or ask a "How do I make my Python code faster" question). If that doesn't help, try to identify the code and port it to C (or Cython) and use the extension.

Answer 7

Pyrex is a subset of the Python language that compiles to C, done by the guy that first built list comprehensions for Python. It was mainly developed for building wrappers but can be used in a more general context.

Answer 8

The answer is "Yes, it is possible". You could take Python code and attempt to compile it into the equivalent C code using the CPython API. In fact, there used to be a Python2C project that did just that, but I haven't heard about it in many years (back in the Python 1.5 days is when I last saw it.)

You could attempt to translate the Python code into native C as much as possible, and fall back to the CPython API when you need actual Python features. I've been toying with that idea myself the last month or two. It is, however, an awful lot of work, and an enormous amount of Python features are very hard to translate into C: nested functions, generators, anything but simple classes with simple methods, anything involving modifying module globals from outside the module, etc, etc.