Accessing C header magic numbers/flags with Cython

Question

Some standard C libraries that I want to access with Cython have a ton of flags. The Cython docs state that I must replicate the parts of the header I need. Which is fine when it comes to functions definitions. They are usually replicated everywhere, docs included. But what about all those magic numbers?

If I want to call mmap, I can always find the function definition and paste it into a .pxd file:

void *mmap(void *start, size_t length, int prot, int flags, int fd, off_t offset)

But calling it needs a ton of flags like PROT_READ, MAP_ANONYMOUS and so on. I have at least two problems with this:

Firstly, it is annoying work to hunt down exactly where those numbers are defined. In fact I'd rather write a .c file and printf the values I need. Are there any better way of finding the value of a given flag such as PROT_READ?

Secondly, how stable are these numbers? Having extracted all the values I need and hardcoded them into my Cython source, what are the chances that compiling on a different platform has switched around, let's say PROT_READ and PROT_EXEC?

Even if the answer is that there are no good or proper ways to do it, I'd like to hear it. I can always accept that something is cumbersome as long as I know I'm not missing something.

Answer 1

There are several possible alternatives:

1. Use the flags from the Python mmap module.
   • simple
   • only works when there are existing Python bindings
2. Use the Python mmap object in the first place, and hand it over to your Cython code
   • even simpler openening
   • might have some Python overhead
3. Use the code generator of ctypeslib
   • some docs on how to extract constants
   • needs gccxml
4. Just copy the numbers.

That being said, the numbers are very, very stable. If they'd change, each and every C program using mmap would have to be recompiled, as the flags from the headers are contained in the binary.

EDIT: mmap is part of POSIX, but a cursory read hasn't revealed whether the flags have to be the same value on all platforms.

Answer 2

Write a file foo.c with this as the contents:

#include <sys/mman.h>

Then run

cpp -dM foo.c | grep -v __ | awk '{if ($3) print $2, "=", $3}' > mman.py

which will create a python file that defines all the constants from mman.h

Obviously, you can do that for multiple includes if you want.

The resulting file might need a bit of cleaning up, but it'll get you close.

Answer 3

To use these constants from Cython, you don't need to figure exactly where they came from or what they are any more than you do from C. For example, your .pxd file can look like

cdef extern from "foo.h":
    void *mmap(void *start, size_t length, int prot, int flags, int fd, off_t offset)
    cdef int PROT_READ
    cdef int MAP_ANONYMOUS
    ...

As long as the definitions are (directly or indirectly) included from foo.h, this should work fine.