How to create make .so files from code written in C or C++ that are usable from Python

Question

Looking at Python modules and at code in the "lib-dnyload" directory in the Python framework, I noticed whenever code is creating some kind of GUI or graphic it imports a non-Python file with a .so extension. And there are tons .so files in "lib-dnyload".

From googling things I found that these files are called shared objects and are written in C or C++. I have a Mac and I use GCC. How do I make shared object files that are accessible via Python? Mainly just how to make shared objects with GCC using Mac OS X.

Answer 1

Official python documentation starts with simple example. It's good enough for start.

Answer 2

In my opinion the easiest way is to use Cython. Cython will generate some C code for you, compile it to make a ".so" library that you can load from python. This is easy and painless. I suggest you to follow the tutorial and you should have a so library pretty soon.

If you want to know all the details behind C extensions of python, you should dive in the Python documentation about C extensions.

Answer 3

You can write python extensions in many ways, including Cython, SWIG, boost.python ... You can also write a shared library and use the "ctypes" library to access it.

Answer 4

The standard way to build and install C extension modules with Python is to use the features of Distutils which is included in the standard Python library. Generally that means including the C file(s) in your project directory and creating a configuration file, normally called setup.py, to identify the C source files. By running a properly-configured setup.py file under Python, Distutils will take care of compiling and linking the C source files by using your system's C compiler and linker. (On OS X, you'll need to have installed Apple's free Xcode Developer Tools which installs versions of gcc.) Wading through all the Distutils documentation to figure out what to do can be overwhelming right now. Here's a simple tutorial that should be enough to get you started.

Why use a Distutils setup.py rather than just calling gcc yourself? Several reasons:

1. Since C extensions normally run within the context of the Python interpreter itself, it is important that extensions be compiled and linked in a manner compatible with the C code in the interpreter. Distutils tries to take the guesswork out of this by supplying the right options for the Python you are executing under. This can be particularly important on OS X where Pythons come in various flavors including some with various combinations of multi-architecture executables (i.e. -arch i386, -arch ppc, -arch x86_64, -arch ppc64) and with support for multiple OS X versions (i.e. the current python.org OS X installers are compatible with OS X 10.4 through 10.6).

2. Distutils allows you to package up and distribute combinations of pure Python and C extension modules in a machine and operating system independent way. In many cases, if the C code is written to avoid operating system dependent system calls and the like, your C extensions can be built and installed on most current Python platforms without modification and without you having to know which C compiler to use or options to set.

3. You'll find that nearly all modern Python third-party packages with C code work this way so it's good to get in the habit of using Distutils for this from the start.

4. As you'll see in the tutorial, it's very easy to setup a setup.py file for many cases.

Answer 5

There are different ways to do this:

• Use the Python.h header in your C code. This produces the fastest code.
• Use wrappers around the Python headers like SWIG. The code may be a bit slower but you need no or minimal change in the C/C++ source code. In most of case you just need to write a SWIG interface files and the shared objects are integrated with Python/PHP/Java etc. It is the easiest way to start to wrapp your C/C++ code.
• If you are more familiar with Python language and want to avoid writing SWIG interface files in C/C++ style than Cython is the best wrapper for you.