How to create Python module distribution to gracefully fall-back to pure Python code

Question

I have written a Python module, and I have two versions: a pure Python implementation and a C extension. I've written the __init__.py file so that it tries to import the C extension, and if that fails, it imports the pure Python code (is that reasonable?).

Now, I'd like to know what is the best way to distribute this module (e.g. write setup.py) so it can be easily used by people with or without the facility to build, or use, the C extension, just by running:

python setup.py install

My experience is limited, but I see two possible cases:

• User does not have MS Visual Studio, or the GCC compiler suite, installed on their machine, to build the C extension
• User is running IronPython, Jython, or anything other than CPython. I only have used CPython. So I'm not sure how I could distribute this module so that it would work smoothly and be easy to install on those platforms, if they're unable to use the C extension.

Answer 1

(is that reasonable?).

Yep, perfectly sensible.

To catch the "no suitable C compiler case": the call to setup(...) will do a sys.exit in case of problems. So, first try it with the ext_modules argument set as desired, within a try:

try:
  setup(..., ext_modules=...)
except SystemExit: ...

and in the except clause, call setup(...) again without the ext_modules (so it gives up on building and installing the extensions). The user who's installing will still see messages like "unable to execute gcc-4.0: No such file or directory", but you can appropriately add your own messages to inform the user that it's no big deal and that you're trying again without the extension modules.

To support non-CPython implementations, in your setup.py you can test sys.version (I'm not sure what the value will be for each non-CPython implementation, but IronPython has an 'IronPython' substring there, for example), to avoid even trying the ext_modules part. If you miss some such implementation in your checks, the try/except should probably catch most others anyway, just with a modest amount of wasted work;-).

Answer 2

"tries to import the C extension, and if that fails, it imports the pure Python code (is that reasonable?)."

Almost. Read about cStringIO and StringIO. Also read about cPickle and Pickle. Also read about cElementTree and ElementTree.

If the C version can't be built, that's one use case. The pure Python version is the only one available.

If the C version can be built, however, I still have good reasons for declining it. Primary, I would consider declining the C version because it may not allow the depth of subclass that I require for my application.

I don't want to be forced to use the C version, just because I happened to have the correct compiler. I prefer to make those decisions on my own.

Consequently, I don't like the idea of some part of your module making my architecture decisions for me. I prefer to choose which to import. If the C version doesn't exist, that doesn't change my decision process, because I may still be creating subclasses of the pure Python version.

Bottom line. Automate less. Provide the two modules. I prefer to choose which one to import.

Answer 3

According to the documentation for Planar, you can make the setup.py file for building the C extensions as normal, and then:

To build and install Planar from the source distribution or repository use:

python setup.py install

To install only the pure-Python modules without compiling, use:

python setup.py build_py install --skip-build