Boost.Python function pointers as class constructor argument

Question

I have a C++ class that requires a function pointer in it's constructor (float(*myfunction)(vector<float>*))
I've already exposed some function pointers to Python.
The ideal way to use this class is something like this:

import mymodule
mymodule.some_class(mymodule.some_function)

So I tell Boost about this class like so:

class_<SomeClass>("some_class", init<float(*)(vector<float>*)>);

But I get:

error: no matching function for call to 'register_shared_ptr1(Sample (*)(std::vector<double, std::allocator<double> >*))'

when I try to compile it.

So, does anyone have any ideas on how I can fix the error without losing the flexibility gained from function pointers (ie no falling back to strings that indicate which function to call)?

Also, the main point of writing this code in C++ is for speed. So it would be nice if I was still able to keep that benefit (the function pointer gets assigned to a member variable during initialization and will get called over a million times later on).

Answer 1

OK, so this is a fairly difficult question to answer in general. The root cause of your problem is that there really is no python type which is exactly equivalent to a C function pointer. Python functions are sort-of close, but their interface doesn't match for a few reasons.

Firstly, I want to mention the technique for wrapping a constructor from here: http://wiki.python.org/moin/boost.python/HowTo#namedconstructors.2BAC8factories.28asPythoninitializers.29. This lets you write an __init__ function for your object that doesn't directly correspond to an actual C++ constructor. Note also, that you might have to specify boost::python::no_init in the boost::python::class_ construction, and then def a real __init__ function later, if your object isn't default-constructible.

Back to the question: Is there only a small set of functions that you'll usually want to pass in? In that case, you could just declare a special enum (or specialized class), make an overload of your constructor that accepts the enum, and use that to look up the real function pointer. You can't directly call the functions yourself from python using this approach, but it's not that bad, and the performance will be the same as using real function pointers.

If you want to provide a general approach that will work for any python callable, things get more complex. You'll have to add a constructor to your C++ object that accepts a general functor, e.g. using boost::function or std::tr1::function. You could replace the existing constructor if you wanted, because function pointers will convert to this type correctly.

So, assuming you've added a boost::function constructor to SomeClass, you should add these functions to your python wrapping code:

struct WrapPythonCallable
{
    typedef float * result_type;
    explicit WrapPythonCallable(const boost::python::object & wrapped) 
        : wrapped_(wrapped)
    { }
    float * operator()(vector<float>* arg) const
    {
        //Do whatever you need to do to convert into a 
        //boost::python::object here
        boost::python::object arg_as_python_object = /* ... */;

        //Call out to python with the object - note that wrapped_ 
        //is callable using an operator() overload, and returns 
        //a boost::python::object.
        //Also, the call can throw boost::python::error_already_set - 
        //you might want to handle that here.
        boost::python::object result_object = wrapped_(arg_as_python_object);

        //Do whatever you need to do to extract a float * from result_object,
        //maybe using boost::python::extract
        float * result = /* ... */;
        return result;
    }
    boost::python::object wrapped_;
};

//This function is the "constructor wrapper" that you'll add to SomeClass.
//Change the return type to match the holder type for SomeClass, like if it's
//held using a shared_ptr.
std::auto_ptr<SomeClass> CreateSomeClassFromPython(
    const boost::python::object & callable)
{
    return std::auto_ptr<SomeClass>(
        new SomeClass(WrapPythonCallable(callable)));
}


//Later, when telling Boost.Python about SomeClass:
class_<SomeClass>("some_class", no_init)
    .def("__init__", make_constructor(&CreateSomeClassFromPython));

I've left out details on how to convert pointers to and from python - that's obviously something that you'll have to work out, because there are object lifetime issues there.

If you need to call the function pointers that you'll pass in to this function from Python, then you'll need to def these functions using Boost.Python at some point. This second approach will work fine with these def'd functions, but calling them will be slow, because objects will be unnecessarily converted to and from Python every time they're called.

To fix this, you can modify CreateSomeClassFromPython to recognize known or common function objects, and replace them with their real function pointers. You can compare python objects' identity in C++ using object1.ptr() == object2.ptr(), equivalent to id(object1) == id(object2) in python.

Finally, you can of course combine the general approach with the enum approach. Be aware when doing this, that boost::python's overloading rules are different from C++'s, and this can bite you when dealing with functions like CreateSomeClassFromPython. Boost.Python tests functions in the order that they are def'd to see if the runtime arguments can be converted to the C++ argument types. So, CreateSomeClassFromPython will prevent single-argument constructors def'd later than it from being used, because its argument matches any python object. Be sure to put it after other single-argument __init__ functions.

If you find yourself doing this sort of thing a lot, then you might want to look at the general boost::function wrapping technique (mentioned on the same page with the named constructor technique): http://wiki.python.org/moin/boost.python/HowTo?action=AttachFile&do=view&target=py_boost_function.hpp.