Python extensions that can be used in all varieties of python (jython / IronPython / etc.)

Question

In the 'old days' when there was just cpython, most extensions were written in c (as platform independent as possible) and compiled into pyd's (think PyCrypto for example). Now there is Jython, IronPython and PyPy and the pyd’s do not work with any of them (Ironclad aside). It seems they all support ctypes and that the best approach MIGHT be to create a platform independent dll or shared library and then use ctypes to interface to it.

But I think this approach will be a bit slower than the old fashion pyd approach. You could also program a pyd for cpython, a similar c# dll for IronPython and a java class or jar for Jython (I'm not sure about PyPy. But while this approach will appeal to platform purists it is very labor intensive. So what is the best route to take today?

Answer 1

Currently, it seems the ctypes is indeed the best approach. It works today, and it's so convenient that it's gonna conquer (most of) the world.

For performance-critical APIs (such as numpy), ctypes is indeed problematic. The cleanest approach would probably be to port Cython to produce native IronPython / Jython / PyPy extensions.

I recall that PyPy had plans to compile ctypes code to efficient wrappers, but as far as I google, there is nothing like that yet...

Answer 2

If you're wrapping an existing native library, the ctypes is absolutely the way to go.

If you're trying to speed up the hot spots in a Python extension, then making a custom extension for each interpreter (and a pure-Python fallback) is tractable because the bulk of the code is pure Python that can be shared, but undesirable and labour-intensive, as you said. You could use ctypes in this case as well.