Contructing python function callable from C , with input parameter having *output* sematics

Question

The use case is the following:

• Given a (fixed, not changeable) DLL implemented in C
• Wanted: a wrapper to this DLL implemented in python (chosen method: ctypes)

Some of the functions in the DLL need synchronization primitives. To aim for maximum flexibility, the designers of the DLL completely rely on client-provided callbacks. More precisely this DLL shall have:

• a callback function to create a synchronizaton object
• callback functions to acquire/release a lock on the synchronizaton object
• and one callback function to destroy the synchronizaton object

Because from the viewpoint of the DLL, the synchronizaton object is opaque, it will be repesented by a void * entity. For example if one of the DLL functions wants to acquire a lock it shall do:

void* mutex;

/* get the mutex object via the create_mutex callback */
create_mutex(&mutex);

/* acquire a lock */
lock_mutex(mutex);

... etc

It can be seen, that the callback create_mutex input parameter has output semantics. This is achieved with void ** signature.

This callback (and the other three) must be implemented in python. I've failed :-) For simplicity, let's focus on only the creating callback, and also for simplicity, let the opaque object be an int.

The toy-DLL, which emulates the use of callbacks, is the following (ct_test.c):

#include <stdio.h>
#include <stdlib.h>

typedef int (* callback_t)(int**);
callback_t func;
int* global_i_p = NULL;

int mock_callback(int** ipp)
{
 int* dynamic_int_p = (int *) malloc(sizeof(int));
 /* dynamic int value from C */
 *dynamic_int_p = 2;
 *ipp = dynamic_int_p;
 return 0;
}

void set_py_callback(callback_t f)
{
 func = f;
}

void set_c_callback()
{
 func = mock_callback;
}

void test_callback(void)
{
 printf("global_i_p before: %p\n", global_i_p);
 func(&global_i_p);
 printf("global_i_p after: %p, pointed value:%d\n", global_i_p, *global_i_p);

 /* to be nice */
 if (func == mock_callback)
  free(global_i_p);
}

The python code, which would like to provide the callback, and use the DLL is the following:

from ctypes import *

lib = CDLL("ct_test.so")

# "dynamic" int value from python
int  = c_int(1)
int_p = pointer(int)

def pyfunc(p_p_i):
 p_p_i.contents = int_p

# create callback type and instance
CALLBACK = CFUNCTYPE(c_int, POINTER (POINTER(c_int)))
c_pyfunc = CALLBACK(pyfunc)

# functions from .so
set_py_callback = lib.set_py_callback
set_c_callback = lib.set_c_callback
test_callback = lib.test_callback

# set one of the callbacks
set_py_callback(c_pyfunc)
#set_c_callback()

# test it
test_callback()

When using the in-DLL provided callback (set via set_c_callback()), this works as expected:

~/dev/test$ python ct_test.py
global_i_p before: (nil)
global_i_p after: 0x97eb008, pointed value:2

However, in the other case - with the python callback - fails:

~/dev/test$ python ct_test.py
global_i_p before: (nil)
Traceback (most recent call last):
  File "/home/packages/python/2.5/python2.5-2.5.2/Modules/_ctypes/callbacks.c", line 284, in 'converting callback result'
TypeError: an integer is required
Exception  in <function pyfunc at 0xa14079c> ignored
Segmentation fault

Where am I wrong?

Answer 1

You appear to be incorrectly defining the return type. It looks like your C callback returns an int, while the Python one you are declaring as return c_int, yet not explicitly returning anything (thus actually returning None). If you "return 0" it might stop crashing. You should do that or change the callback signature to CFUNCTYPE(None, ...etc) in any case.

Also, although it's not a current problem here, you're shadowing the "int" builtin name. This might lead to problems later.

Edited: to correctly refer to the C return type as "int", not "void".

Answer 2

The segfault is due to incorrect pointer handling in your Python callback. You have more levels of pointer indirection than strict necessary, which is probably the source of your confusion. In the Python callback you set p_p_i.contents, but that only changes what the Python ctypes object points at, not the underlying pointer. To do that, do pointer derefrence via array access syntax. A distilled example:

ip = ctypes.POINTER(ctypes.c_int)()
i = ctypes.c_int(99)
# Wrong way
ipp = ctypes.pointer(ip)
ipp.contents = ctypes.pointer(i)
print bool(ip) # False --> still NULL
# Right way
ipp = ctypes.pointer(ip)
ipp[0] = ctypes.pointer(i)
print ip[0] # 99 --> success!

The type error is due to a type incompatibility as described in Peter Hansen's answer.