Creating an inheritable Python type with PyCxx

Question

A friend and I have been toying around with various Python C++ wrappers lately, trying to find one that meets the needs of both some professional and hobby projects. We've both honed in on PyCxx as a good balance between being lightweight and easy to interface with while hiding away some of the ugliest bits of the Python C api. PyCxx is not terribly robust when it comes to exposing types, however (ie: it instructs you to create type factories rather than implement constructors), and we have been working on filling in the gaps in order to expose our types in a more functional manner. In order to fill these gaps we turn to the C api.

This has left us with some questions, however, that the api documentation doesn't seem to cover in much depth (and when it does, the answers are occasionally contradictory). The basic overarching question is simply this: What must be defined for a Python type to function as a base type? We've found that for the PyCxx class to function as a type we need to define tp_new and tp_dealloc explicitly and set the type as a module attribute, and that we need to have Py_TPFLAGS_BASETYPE set on [our type]->tp_flags, but beyond that we're still groping in the dark.

Here is our code thus far:

class kitty : public Py::PythonExtension<kitty> {
public:
    kitty() : Py::PythonExtension<kitty>() {}
    virtual ~kitty() {}

    static void init_type() {
     behaviors().name("kitty");
     add_varargs_method("speak", &kitty::speak);
    }

    static PyObject* tp_new(PyTypeObject *subtype, PyObject *args, PyObject *kwds) {
     return static_cast<PyObject*>(new kitty());
    }

    static void tp_dealloc(PyObject *obj) {
     kitty* k = static_cast<kitty*>(obj);
     delete k;
    }

private:
    Py::Object speak(const Py::Tuple &args) {
     cout << "Meow!" << endl;
     return Py::None();
    }
};

// cat Module
class cat_module : public Py::ExtensionModule<cat_module> {
public: 
    cat_module() : Py::ExtensionModule<cat_module>("cat") {

     kitty::init_type();

     // Set up additional properties on the kitty type object
     PyTypeObject* kittyType = kitty::type_object();
     kittyType->tp_new = &kitty::tp_new;
     kittyType->tp_dealloc = &kitty::tp_dealloc;
     kittyType->tp_flags |= Py_TPFLAGS_BASETYPE;

     // Expose the kitty type through the module
     module().setAttr("kitty", Py::Object((PyObject*)kittyType));

     initialize();
    }
    virtual ~cat_module() {}
};

extern "C" void initcat() {
    static cat_module* cat = new cat_module();
}

And our Python test code looks like this:

import cat

class meanKitty(cat.kitty):
    def scratch(self):
     print "hiss! *scratch*"

myKitty = cat.kitty()
myKitty.speak()

meanKitty = meanKitty()
meanKitty.speak()
meanKitty.scratch()

The curious bit is that if you comment all the meanKitty bits out, the script runs and the cat meows just fine, but if you uncomment the meanKitty class suddenly Python gives us this:

AttributeError: 'kitty' object has no attribute 'speak'

Which confuses the crap out of me. It's as if inheriting from it hides the base class entirely! If anyone could provide some insight into what we are missing, it would be appreciated! Thanks!

---

EDIT: Okay, so about five seconds after posting this I recalled something that we had wanted to try earlier. I added the following code to kitty -

virtual Py::Object getattr( const char *name ) {
    return getattr_methods( name );
}

And now we're meowing on both kitties in Python! still not fully there, however, because now I get this:

Traceback (most recent call last):
    File "d:\Development\Junk Projects\PythonCxx\Toji.py", line 12, in <module>
    meanKitty.scratch()
AttributeError: scratch

So still looking for some help! Thanks!

Answer 1

I've only done a tiny bit of work with PyCxx, and I'm not at a compiler, but I suspect what you're seeing is similar to the following situation, as expressed in pure Python:

>>> class C(object):
...   def __getattribute__(self, key):
...     print 'C', key
... 
>>> class D(C):
...   def __init__(self):
...     self.foo = 1
... 
>>> D().foo
C foo
>>>

My best guess is that the C++ getattr method should check this.ob_type->tp_dict (which will of course be the subclass' dict, if this an instance of the subclass) and only call getattr_methods if you fail to find name in there (see the PyDict_ API functions).

Also, I don't think you should set tp_dealloc yourself: I don't see how your implementation improves on PyCxx's default extension_object_deallocator.

Answer 2

You must declare kitty as class new_style_class: public Py::PythonClass< new_style_class >. See simple.cxx and the Python test case at http://cxx.svn.sourceforge.net/viewvc/cxx/trunk/CXX/Demo/Python3/.

Python 2.2 introduced new-style classes which among other things allow the user to subclass built-in types (like your new built-in type). Inheritance didn't work in your example because it defines an old-style class.