Boost.Python - How to return by reference?

Question

I'm using Boost.Python to create Python modules from C++ classes. And I ran into a problem with references.

Condider the following case where I have a class Foo with overloaded get methods that can either return by value or reference.

Specifying that the return by value should be used was easy once I typedefed a signature. But I think it should be possible return a reference as well by using a return_value_policy. However, using what seemed appropriate (doc); return_value_policy<reference_existing_object> did not seem to work.

Have I misunderstood what it does?

struct Foo {
    Foo(float x) { _x = x; }
    float& get() { return _x; }
    float  get() const { return _x; }
private:
    float _x;
};

// Wrapper code
BOOST_PYTHON_MODULE(my_module)
{
    using namespace boost::python;
    typedef float (Foo::*get_by_value)() const;
    typedef float& (Foo::*get_by_ref)();

    class_<Foo>("Foo", init<float>())
        .def("get", get_by_value(&Foo::get))
        .def("get_ref", get_by_ref(&Foo::get),
            return_value_policy<reference_existing_object>())//Doesn't work
        ;
}

Note: I know it could be dangerous to reference existing object without life-time managing.

Update:
It looks like it works for objects but not basic data types.
Take this revised example:

struct Foo {
    Foo(float x) { _x = x; }
    float& get() { return _x; }
    float  get() const { return _x; }
    void set( float f ){ _x = f;}
    Foo& self(){return *this;}
private:
    float _x;
};

// Wrapper code
using namespace boost::python;
BOOST_PYTHON_MODULE(my_module)
{
    typedef float (Foo::*get_by_value)() const;

    class_<Foo>("Foo", init<float>())
        .def("get", get_by_value(&Foo::get))
        .def("get_self", &Foo::self,
            return_value_policy<reference_existing_object>())
        .def("set", &Foo::set);
        ;
}

Which in a test gave the expected result:

>>> foo1 = Foo(123)
>>> foo1.get()
123.0
>>> foo2 = foo1.get_self()
>>> foo2.set(1)
>>> foo1.get()
1.0
>>> id(foo1) == id(foo2)
False

Answer 1

I don't know much about Boost.Python, so I may misunderstand the question, in which case this is completely unhelpful. But here goes:

In Python you can't choose between returning by reference or by value, the distinction doesn't make sense in Python. I find it's easiest to think of it as everything being handled by reference.

You just have objects, and you have names for those objects. So

foo = "ryiuy"

Creates the string object "ryiuy" and then lets you refer to that string object with the name "foo". So in Python, when you get passed something, you get passed that object. There is no "values" as such, so you can't pass the value. But then again, it's also a valid viewpoint that there aren't references either, just objects and their names.

So the answer is, I guess, is that when you get a reference in C, you need to pass a reference to the object that reference references into Python. And when you get a value in C, you need to pass a reference to the object that you create from that value into Python.

Answer 2

Are you sure that the c++ object is being copied? You will get a new python object each time but which references the same c++ object. How are you determining that the object has been copied?

Answer 3

I think you want return internal reference instead. I have used it before to do something similar.

Answer 4

In Python, there's the concept of immutable types. An immutable types can't have its value changed. Examples of built-in immutable types are int, float and str.

Having said that, you can't do what you want with boost::python, because Python itself does not allow you to change the value of the float returned by the function.

Your second example shows one solution, another would be to create thin-wrappers and exposing that:

void Foo_set_x(Foo& self, float value) {
    self.get() = value;
}

class_<Foo>("Foo", init<float>())
    ...
    .def("set", &Foo_set_x);
;

Which is a better solution than having to change the original C++ class.