Python decorators compared to CLOS "around" method.

Question

Hi, I'm reaching back to my CLOS (Common Lisp Object System) days for this abstract question.

I'm augmenting the question to clarify:

It appears to me that a Python decorator is sort of like an "around" method in CLOS.

From what I remember, an "around" method in CLOS is a method/function that wraps around the primary method/function of the same name. It traverses up and down sub-classes too. Here's some syntax (I just grabbed my book).

All of these methods This would be inside a class:

(defmethod helloworld ()
  (format t "Hello World"))

There can be before and after methods too (which I'm throwing in for completeness):

(defmethod helloworld :before ()
  (format t "I'm executing before the primary-method"))

(defmethod helloworld :after ()
  (format t "I'm executing after the primary-method"))

And finally the around method (Notice here that this method seemed to be like a decorator):

(defmethod helloworld :around ()
  (format t "I'm the most specific around method calling next method.")
  (call-next-method)
  (format t "I'm the most specific around method done calling next method."))

I believe the output would be:

I'm the most specific around method calling next method. 
I'm executing before the primary-method
Hello World
I'm executing after the primary-method
I'm the most specific around method done calling next method.

I have always used this understanding of classes and their methods as a reference point for developing code. And unfortunately few languages seem to get this far with their method parameterization and power.

I'm pretty new to Python and am trying to see how decorators fit in. They seem a little looser in that a decorator can be a completely external function which yet has the ability to manipulate information within the calling information and even modifying the instance and class variables of the object called, and further that it seems to preform the role of the around method as shown here. But I was hoping somebody could help explain the relationship between decorators and around methods. I thought somebody would really like the opportunity to do that.

What makes CLOS powerful to me is that you can have multiple inheritance with these methods. Thus a class can be made up of superclasses that contain distinct functionalities and attributes which handle themselves. Thus an around method on one of the superclasses might terminate flow (if "call-next-method" is not done), just as the way a decorator can apparently work. So is this the same as a decorator, or different? In an around method, you're passing in the same arguments, but to a decorator, you're passing in the "function" in a strict definition which gets augmented. But is the outcome the same?

Thanks much! Maybe somebody could show closes approximation to the above in Python. done calling next method.

So the issue is not about implementing the CLOS methods in Python, but showing how close Python gets to that system in a pythonic way. Or showing how Python is actually better than that.

This is more of the kind of example I was thinking of:

class shape with attributes position and method area
class renderable with attribute visible and methods render, and render :around
class triangle with superclass shape and renderable attributes p1,p2,p3 and method render and method area
class square ...

If an instance of triangle has visible=false, then the render :around will not call the triangle's primary method.

In other words the calling chain of the render method is (a) renderable :around, (b) triangle primary, (c) finish renderable :around. If triangle had an :after method, it would be called after primary, and then the around method would finish up.

I understand the difficulties of using inheritance versus considering newer design patterns but here I'm trying to bridge my CLOS knowledge. If there's a design pattern that matches decorators (more accurately than the "decorator" design pattern), that would be great to understand also.

Conclusions

I'm getting the hang of decorators. But I wanted to present where I'm at with trying to emulate the CLOS method traversal. Everybody inspired me to try it since I've got the book and I remember it pretty well. Thanks all for all the great suggestions, they're all a piece of the puzzle. In terms of implementing the actual structure in a single decorator, Will got close and that's what worked for moving it forward with dynamic method finding (see below). I've created a single decorator that does what I'm looking for and can operate on any class. I'm sure it could be cleaner and there's a problem that it only looks up one superclass and it's doing around methods weirdly, but it does work.

'''file: cw.py'''
'''This decorator does the job of implementing a CLOS method traversal through superclasses.  It is a very remedial example but it helped me understand the power of decorators.'''
'''Modified based on Richards comments'''
def closwrapper(func): # *args, **kwargs  ?
    def wrapper(self):  #what about superclass traversals???
        name = func.__name__
        # look for the methods of the class 
        before_func = getattr(self, name + "_before", None)
        after_func = getattr(self, name + "_after", None)
        around_func = getattr(self, name + "_around", None)
        sup = super(self.__class__,self)
        #self.__class__.__mro__[1]
        if sup:
            # look for the supermethods of the class (should be recursive)
            super_before_func = getattr(sup,name + "_before", None)
            super_after_func = getattr(sup,name + "_after", None))
            super_around_func = getattr(sup,name + "_around", None))

        ''' This is the wrapper function which upgrades the primary method with any other methods that were found above'''
        ''' The decorator looks up to the superclass for the functions.  Unfortunately, even if the superclass is decorated, it doesn't continue chaining up.  So that's a severe limitation of this implementation.'''
        def newfunc():
            gocontinue = True
            supercontinue = True
            if around_func: 
                gocontinue = around_func() 
                if gocontinue and super_around_func:
                  supercontinue = super_around_func()
            if gocontinue and supercontinue:
                if before_func: before_func()
                if super_before_func: super_before_func()
                result = func(self)
                if super_after_func: super_after_func()   
                if after_func: after_func()              
            else:
                result = None
            if gocontinue:
                if super_around_func: super_around_func(direction="out")
            if around_func: around_func(direction='out')
            return result
        return newfunc()

    return wrapper

# Really, the way to do this is to have the decorator end up decorating
# all the methods, the primary and the before and afters.  Now THAT would be a decorator!

class weeclass(object):

    @closwrapper
    def helloworld(self):
        print "Hello Wee World"

    def helloworld_before(self):
        print "Am I really so wee Before?  This method is not called on subclass but should be"



class baseclass(weeclass):
    fooey = 1

    def __init__(self):
        self.calls = 0

    @closwrapper
    def helloworld(self):
        print "Hello World"

    def helloworld_before(self):
        self.fooey += 2
        print "Baseclass Before"

    def helloworld_after(self):
        self.fooey += 2
        print "Baseclass After Fooey Now",self.fooey

    def helloworld_around(self,direction='in'):
        if direction=='in': 
            print "Aound Start"
            if self.fooey < 10:
                return True
            else:
                print ">>FOOEY IS TOO BIG!!!"
                self.fooey = -10
                return False
        #call-next-method
        if not direction=='in': 
            #self.barrey -= 4  #hello??  This should not work!!!  It should croak?
            print "Around End"  



class subclass(baseclass): 
    barrey = 2

    @closwrapper
    def helloworld(self):
        print "Hello Sub World Fooey",self.fooey,"barrey",self.barrey

    def helloworld_before(self):
        self.fooey -= 1
        self.barrey += 5
        print "  Sub Before"

    def helloworld_after(self):
        print "Sub After"

    def helloworld_around(self,direction='in'):
        if direction=='in': 
            print "Sub Around Start"
            if self.barrey > 4:
                print ">>Hey Barrey too big!"
                self.barrey -= 8
                return False
            else:
                return True
        #call-next-method
        if not direction=='in': 
            self.barrey -= 4
            print "Sub Around End"  

Here is the output so you can see what I'm trying to do.

Python 2.6.4 (r264:75706, Mar  1 2010, 12:29:19)  
[GCC 4.2.1 (Apple Inc. build 5646) (dot 1)] on darwin  
Type "help", "copyright", "credits" or "license" for more information.  
>>> import cw  
>>> s= cw.subclass()  
>>> s.helloworld()  
Sub Around Start  
Aound Start  
Sub Before  
Baseclass Before  
Hello Sub World Fooey 2 barrey 7  
Baseclass After Fooey Now 4  
Sub After  
Around End  
Sub Around End  
>>> s.helloworld()  
Sub Around Start
Aound Start  
Sub Before  
Baseclass Before  
Hello Sub World Fooey 5 barrey 8  
Baseclass After Fooey Now 7  
Sub After  
Around End  
Sub Around End  
>>> s.helloworld()  
Sub Around Start  
Aound Start  
Sub Before  
Baseclass Before  
Hello Sub World Fooey 8 barrey 9  
Baseclass After Fooey Now 10  
Sub After  
Around End  
Sub Around End  
>>> s.helloworld()  
Sub Around Start  
>>Hey Barrey too big!  
Sub Around End  
>>> s.helloworld()  
Sub Around Start  
Aound Start  
>>FOOEY IS TOO BIG!!!  
Around End  
Sub Around End  
>>> s.helloworld()  
Sub Around Start  
Aound Start  
Sub Before  
Baseclass Before  
Hello Sub World Fooey -9 barrey -6  
Baseclass After Fooey Now -7  
Sub After  
Around End  
Sub Around End  
>>> s.helloworld()  
Sub Around Start  
Aound Start  
Sub Before  
Baseclass Before  
Hello Sub World Fooey -6 barrey -5  
Baseclass After Fooey Now -4  
Sub After  
Around End  
Sub Around End  
>>> s.helloworld()  
Sub Around Start  
Aound Start  
Sub Before  
Baseclass Before  
Hello Sub World Fooey -3 barrey -4  
Baseclass After Fooey Now -1  
Sub After  
Around End  
Sub Around End  
>>> b = cw.baseclass()  
>>> b.helloworld()  
Aound Start  
Baseclass Before  
Am I really so wee Before?  This method is not called on subclass but should be  
Hello World  
Baseclass After Fooey Now 5  
Around End  
>>> b.helloworld()  
Aound Start  
Baseclass Before  
Am I really so wee Before?  This method is not called on subclass but should be  
Hello World  
Baseclass After Fooey Now 9  
Around End  
>>> b.helloworld()  
Aound Start  
Baseclass Before  
Am I really so wee Before?  This method is not called on subclass but should be  
Hello World  
Baseclass After Fooey Now 13  
Around End  
>>> b.helloworld()  
Aound Start  
>>FOOEY IS TOO BIG!!!  
Around End  
>>> b.helloworld()  
Aound Start  
Baseclass Before  
Am I really so wee Before?  This method is not called on subclass but should be  
Hello World  
Baseclass After Fooey Now -6  
Around End  

I hope that creates some understand of the CLOS calling and also sparks ideas on how to improve that decorator, or how to lambast me for even trying to do it. :-)

Answer 1

Here's a quick and dirty implementation slightly better implementation (now with the around method called hopefully in the right places), using decorators

def hints(before=None, after=None, around=None):
    """A decorator that implements function hints to be run before, after or
    around another function, sort of like in the CLOS."""

    # Make sure all of our hints are callable
    default = lambda *args, **kwargs: None
    before = before if callable(before) else default
    after = after if callable(after) else default
    around = around if callable(around) else default

    # The actual decorator function.  The "real" function to be called will be
    # pased to this as `fn`
    def decorator(fn):

        # The decorated function.  This is where the work is done.  The before
        # and around functions are called, then the "real" function is called
        # and its results are stored, then the around and after functions are
        # called.
        def decorated(*args, **kwargs):
            around(*args, **kwargs)
            before(*args, **kwargs)
            result = fn(*args, **kwargs)
            after(*args, **kwargs)
            around(*args, **kwargs)
            return result
        return decorated
    return decorator

# Shortcuts for defining just one kind of hint
def before(hint):
    return hints(before=hint)

def after(hint):
    return hints(after=hint)

def around(hint):
    return hints(around=hint)


# The actual functions to run before, after, around
def beforefn():
    print 'before'

def afterfn():
    print 'after'

def aroundfn():
    print 'around'


# The function around which the other functions should run
@before(beforefn)
@after(afterfn)
@around(aroundfn)
def fn():
    print 'Hello World!'

# Or use the single @hints decorator
@hints(before=beforefn, after=afterfn, around=aroundfn)
def fn2():
    print 'Goodbye World!'

Calling fn() results in this:

>>> fn()
around
before
Hello World!
after
around
>>> fn2()
around
before
Goodbye World!
after
around

The decorators in this case might be a little bit confusing, because there are two nested functions involved in each, rather than the one nested function seen in a lot of decorators.

It might not be as elegant as the CLOS version (and I may be missing some of its functionality), but it seems to do what you want.

Answer 2

I'm not sure I understand :around, :before and :after very well, but is something like this what you are looking for?

class Base(object):
    def helloworld(self):
        print('Hello World')

class After(object):
    def helloworld(self):
        super(After,self).helloworld()
        print('After')        

class Before(object):
    def helloworld(self):
        print('Before')        
        super(Before,self).helloworld()

class Around(object):
    def helloworld(self):
        print('Enter around')
        super(Around,self).helloworld()        
        print('Exit around around')

class Foo(Around,Before,After,Base):
    def helloworld(self):
        super(Foo,self).helloworld()

foo=Foo()

This is foo's MRO (method resolution order).

print([cls.__name__ for cls in foo.__class__.mro()])
# ['Foo', 'Around', 'Before', 'After', 'Base', 'object']

When you say super(cls,self).helloworld(), Python

1. looks at self's MRO
2. finds the next class after cls
3. calls that class's helloworld method

So:

foo.helloworld()

yields

# Enter around
# Before
# Hello World
# After
# Exit around around

For more on super and MRO, see this excellent article by Shalabh Chaturvedi.

Answer 3

You could implement something similar. Will was on the right track, but it seems like "call-next-method" is pretty pivotal to the use of "around", which can be implemented as such:

def around(callback):
  def decorator(fn):
    return lambda *a, **kw: callback(lambda: fn(*a, **kw))
  return decorator

def hello_before(call_next_method):
  print("I'm executing before the primary-method")
  return call_next_method()

def hello_after(call_next_method):
  value = call_next_method()
  print("I'm executing after the primary-method")
  return value


def hello_around(call_next_method):
  print "I'm the most specific around method calling next method."
  value = call_next_method()
  print("I'm the most specific around method done calling next method.")
  return value


@around(hello_around)
@around(hello_after)
@around(hello_before)
def helloworld():
  print("Hello world")

helloworld()

This produces exactly the same output as yours, with reasonably similar constructs. Just pay attention to the order you decorate the function with.