Class Decorators, Inheritance, super(), and maximum recursion

Question

I'm trying to figure out how to use decorators on subclasses that use super(). Since my class decorator creates another subclass a decorated class seems to prevent the use of super() when it changes the className passed to super(className, self). Below is an example:

def class_decorator(cls):
    class _DecoratedClass(cls):
        def __init__(self):
            return super(_DecoratedClass, self).__init__()
    return _DecoratedClass

class BaseClass(object):
    def __init__(self):
        print "class: %s" % self.__class__.__name__
    def print_class(self):
        print "class: %s" % self.__class__.__name__

bc = BaseClass().print_class()

class SubClass(BaseClass):
    def print_class(self):
        super(SubClass, self).print_class()

sc = SubClass().print_class()

@class_decorator
class SubClassAgain(BaseClass):
    def print_class(self):
        super(SubClassAgain, self).print_class()

sca = SubClassAgain()
# sca.print_class() # Uncomment for maximum recursion

The output should be:

class: BaseClass
class: BaseClass
class: SubClass
class: SubClass
class: _DecoratedClass
Traceback (most recent call last):
File "class_decorator_super.py", line 34, in <module>
sca.print_class()
File "class_decorator_super.py", line 31, in print_class
super(SubClassAgain, self).print_class()
...
...
RuntimeError: maximum recursion depth exceeded while calling a Python object

Does anyone know of a way to not break a subclass that uses super() when using a decorator? Ideally I'd like to reuse a class from time to time and simply decorate it w/out breaking it.

Answer 1

Are you sure you want to use a class decorator and not simply inheritance? For instance, instead of a decorator to replace your class with a subclass introducing some methods, perhaps you want a mixin class and to use multiple inheritance to create the final class?

This would be accomplished by something like

class MyMixIn(object):
    def __init__(self):
        super(MyMixIn, self).__init__()

class BaseClass(object):
    def __init__(self):
        print "class: %s" % self.__class__.__name__
    def print_class(self):
        print "class: %s" % self.__class__.__name__

class SubClassAgain(BaseClass, MyMixIn):
    def print_class(self):
        super(SubClassAgain, self).print_class()

sca = SubClassAgain()
sca.print_class() 

Answer 2

Basically, you can see the problem after entering your code sample at the interactive Python prompt:

>>> SubClassAgain
<class '__main__._DecoratedClass'>

i.e., the name SubClassAgain is now bound (in global scope, in this case) to a class that in fact isn't the "real" SubClassAgain, but a subclass thereof. So, any late-bound reference to that name, like the one you have in its super(SubClassAgain, call, will of course get the subclass that's masquerading by that name -- that subclass's superclass is of course "the real SubClassAgain", whence the infinite recursion.

You can reproduce the same problem very simply without decoration, just by having any subclass usurp its base-class's name:

>>> class Base(object):
...   def pcl(self): print 'cl: %s' % self.__class__.__name__
... 
>>> class Sub(Base):
...   def pcl(self): super(Sub, self).pcl()
... 
>>> Sub().pcl()
cl: Sub
>>> class Sub(Sub): pass
... 

now, Sub().pcl() will cause infinite recursion, due to the "name usurpation". Class decoration, unless you use it to decorate and return the same class you get as an argument, is systematic "name usurpation", and thus incompatible with uses of the class name which absolutely must return the "true" class of that name, and not the usurper (be that in self or otherwise).

Workarounds -- if you absolutely must have both class decoration as usurpation (not just class decoration by changes in the received class argument), and super -- basically need protocols for cooperation between the usurper and the possible-usurpee, such as the following small changes to your example code:

def class_decorator(cls):
    class _DecoratedClass(cls):
    _thesuper = cls
        def __init__(self):
            return super(_DecoratedClass, self).__init__()
    return _DecoratedClass

   ...

@class_decorator
class SubClassAgain(BaseClass):
    def print_class(self):
    cls = SubClassAgain
    if '_thesuper' in cls.__dict__:
        cls = cls._thesuper
        super(cls, self).print_class()

Answer 3

As you might already be aware, the problem arises from the fact that the name SubClassAgain in SubClassAgain.print_class is scoped to the current module's global namespace. SubClassAgain thus refers to the class _DecoratedClass rather than the class that gets decorated. One way of getting at the decorated class is to follow a convention that class decorators have a property referring to the decorated class.

def class_decorator(cls):
    class _DecoratedClass(cls):
        original=cls
        def __init__(self):
            print '_DecoratedClass.__init__'
            return super(_DecoratedClass, self).__init__()
    return _DecoratedClass

@class_decorator
class SubClassAgain(BaseClass):
    original
    def print_class(self):
        super(self.__class__.original, self).print_class()

Another is to use the __bases__ property to get the decorated class.

@class_decorator
class SubClassAgain(BaseClass):
    def print_class(self):
        super(self.__class__.__bases__[0], self).print_class()

Of course, with multiple decorators, either of these become unwieldy. The latter also doesn't work with subclasses of the decorated class. You can combine decorators and mixins, writing a decorator that adds a mixin into a class. This won't help you override methods.

def class_decorator(cls):
    class _DecoratedClass(object):
        def foo(self):
            return 'foo'
    cls.__bases__ += (_DecoratedClass, )
    return cls

Lastly, you can work directly with the class attributes to set methods.

def class_decorator(cls):
    old_init = getattr(cls, '__init__')
    def __init__(self, *args, **kwargs):
        print 'decorated __init__'
        old_init(self, *args, **kwargs)
    setattr(cls, '__init__', __init__)
    return cls

This is probably the best option for your example, though the mixin-based decorator also has its uses.

Answer 4

The decorator creates a kind-of diamond inheritance situation. You can avoid these problems by not using super(). Changing SubClassAgain to the following will prevent infinite recursion:

@class_decorator
class SubClassAgain(BaseClass):
    def print_class(self):
        BaseClass.print_class(self)

Answer 5

How about simply promoting _DecoratedClass's __bases__ up to the __bases__ of SubClassAgain?

def class_decorator(cls):
    class _DecoratedClass(cls):
        def __init__(self):
            return super(_DecoratedClass, self).__init__()
    _DecoratedClass.__bases__=cls.__bases__
    return _DecoratedClass