Function Decorators

Question

I like being able to measure performance of the python functions I code, so very often I do something similar to this...

import time

def some_function(arg1, arg2, ..., argN, verbose = True) :
    t = time.clock() # works best in Windows
    # t = time.time() # apparently works better in Linux

    # Function code goes here

    t = time.clock() - t
    if verbose :
        print "some_function executed in",t,"sec."

    return return_val

Yes, I know you are supposed to measure performance with timeit, but this works just fine for my needs, and allows me to turn this information on and off for debugging very smoothly.

That code of course was from before I knew about function decorators... Not that I know much about them now, but I think I could write a decorator that did the following, using the **kwds dictionary:

some_function(arg1, arg2, ..., argN) # Does not time function
some_function(arg1, arg2, ..., argN, verbose = True) # Times function

I would nevertheless like to duplicate the prior working of my functions, so that the working would be something more like:

some_function(arg1, arg2, ..., argN) # Does not time function
some_function(arg1, arg2, ..., argN, False) # Does not time function
some_function(arg1, arg2, ..., argN, True) # Times function

I guess this would require the decorator to count the number of arguments, know how many the original function will take, strip any in excess, pass the right number of them to the function... I'm uncertain though on how to tell python to do this... Is it possible? Is there a better way of achieving the same?

Answer 1

Though inspect may get you a bit on the way, what you want is in general not possible:

def f(*args):
    pass

Now how many arguments does f take? Since *args and **kwargs allow for an arbitrary number of arguments, there is no way to determine the number of arguments a function requires. In fact there are cases where the function really handles as many as there are thrown at it!

---

Edit: if you're willing to put up with verbose as a special keyword argument, you can do this:

import time

def timed(f):
    def dec(*args, **kwargs):
        verbose = kwargs.pop('verbose', False)
        t = time.clock()

        ret = f(*args, **kwargs)

        if verbose:
            print("%s executed in %ds" % (f.__name__, time.clock() - t))

        return ret

    return dec

@timed
def add(a, b):
    return a + b

print(add(2, 2, verbose=True))

(Thanks Alex Martelli for the kwargs.pop tip!)

Answer 2

+1 on Stephan202's answer, however (putting this in a separate answer since comments don't format code well!), the following bit of code in that answer:

verbose = False
if 'verbose' in kwargs:
     verbose = True
     del kwargs['verbose']

can be expressed much more clearly and concisely as:

verbose = kwargs.pop('verbose', False)

Answer 3

it might be difficult but you can do something on these lines. Code below tries to remove any extra arguments and prints them out.

def mydeco(func):
    def wrap(*args, **kwargs):
        """
        we want to eat any extra argument, so just count args and kwargs
        and if more(>func.func_code.co_argcount) first take it out from kwargs 
        based on func.func_code.co_varnames, else last one from args
        """
        extraArgs = []

        newKwargs = {}
        for name, value in kwargs.iteritems():
            if name in func.func_code.co_varnames:
                newKwargs[name] = value
            else:
                extraArgs.append(kwargs[name])

        diff = len(args) + len(newKwargs) - func.func_code.co_argcount
        if diff:
            extraArgs.extend(args[-diff:])
            args = args[:-diff]

        func(*args, **newKwargs)
        print "%s has extra args=%s"%(func.func_name, extraArgs)

    return wrap

@mydeco
def func1(a, b, c=3):
    pass

func1(1,b=2,c=3, d="x")
func1(1,2,3,"y")

output is

func1 has extra args=['x']
func1 has extra args=['y']