tags:

views:

64

answers:

2

I'm considering integrating some C code into a Python system (Django), and I was considering using the Python / C API. The alternative is two separate processes with IPC, but I'm looking into direct interaction first. I'm new to Python so I'm trying to get a feel for the right direction to take.

Is it possible for a call to a C initialiser function to malloc a block of memory (and put something in it) and return a handle to it back to the Python script (pointer to the start of the memory block). The allocated memory should remain on the heap after the init function returns. The Python script can then call subsequent C functions (passing as an argument the pointer to the start of memory) and the function can do some thinking and return a value to the Python script. Finally, there's another C function to deallocate the memory.

Assume that the application is single-threaded and that after the init function, the memory is only read from so concurrency isn't an issue. The amount of memory will be a few hundred megabytes.

  • Is this even possible? Will Python let me malloc from the heap and allow it to stay there? Will it come from the Python process's memory? Will Python try and clear it up (i.e. does it do its own memory allocation and not expect any other processes to interfere with its address space)?
  • Could I just return the byte array as a Python managed string (or similar datatype) and pass the reference back as an argument to the C call? Would Python be OK with such a large string?
  • Would I be better off doing this with a separate process and IPC?
+1  A: 

Cython

Ignacio Vazquez-Abrams
Thanks. The FAQ says that it doesn't officially implement Python. Would I be correct to be a bit wary about running a production Django website on Cython?
Joe
What it means is that it doesn't implement the full Python syntax as given by the grammar file in the source distribution. It still supports enough to be useful, and is almost indistinguishable from a module written in C except that it was much, much easier to write.
Ignacio Vazquez-Abrams
Okay, I'll look into it.
Joe
+1  A: 

You can certainly use the C API to do what you want. You'll create a class in C, which can hold onto any memory it wants. That memory doesn't have to be exposed to Python at all if you don't want.

If you are comfortable building C DLLs, and don't need to perform Python operations in C, then ctypes might be your best bet.

Ned Batchelder
I'm not going to be working on Windows, not sure if that makes any difference.
Joe
(Looks like it's cross platform)
Joe