How can this python function code work?

Question

this is from the source code of csv2rec in matplotlib

how can this function work, if its only parameters are 'func, default'?

def with_default_value(func, default):
    def newfunc(name, val):
        if ismissing(name, val):
            return default
        else:
            return func(val)
    return newfunc

ismissing takes a name and a value and determines if the row should be masked in a numpy array.

func will either be str, int, float, or dateparser...it converts data. Maybe not important. I'm just wondering how it can get a 'name' and a 'value'

I'm a beginner. Thanks for any 2cents! I hope to get good enough to help others!

Answer 1

This is a function that returns another function. name and value are the parameters of the returned function.

Answer 2

This is a Python decorator -- basically a function wrapper. (Read all about decorators in PEP 318 -- http://www.python.org/dev/peps/pep-0318/)

If you look through the code, you will probably find something like this:

def some_func(name, val):
    # ...
some_func = with_default_value(some_func, 'the_default_value')

The intention of this decorator seems to supply a default value if either the name or val arguments are missing (presumably, if they are set to None).

Answer 3

As for why it works:

with_default_value returns a function object, which is basically going to be a copy of that nested newfunc, with the 'func' call and default value substited with whatever was passed to with_default_value.

If someone does 'foo = with_default_value(bar, 3)', the return value is basically going to be a new function:

def foo(name, val):
    ifismissing(name, val):
        return 3
    else:
        return bar(val)

so you can then take that return value, and call it.

Answer 4

This with_default_value function is what's often referred to (imprecisely) as "a closure" (technically, the closure is rather the inner function that gets returned, here newfunc -- see e.g. here). More generically, with_default_value is a higher-order function ("HOF"): it takes a function (func) as an argument, it also returns a function (newfunc) as the result.

I've seen answers confusing this with the decorator concept and construct in Python, which is definitely not the case -- especially since you mention func as often being a built-in such as int. Decorators are also higher-order functions, but rather specific ones: ones which return a decorated, i.e. "enriched", version of their function argument (which must be the only argument -- "decorators with arguments" are obtained through one more level of function/closure nesting, not by giving the decorator HOF more than one argument), which gets reassigned to exactly the same name as that function argument (and so typically has the same signature -- using a decorator otherwise would be extremely peculiar, un-idiomatic, unreadable, etc).

So forget decorators, which have absolutely nothing to do with the case, and focus on the newfunc closure. A lexically nested function can refer to (though not rebind) all local variable names (including argument names, since arguments are local variables) of the enclosing function(s) -- that's why it's known as a closure: it's "closed over" these "free variables". Here, newfunc can refer to func and default -- and does.

Higher-order functions are a very natural thing in Python, especially since functions are first-class objects (so there's nothing special you need to do to pass them as arguments, return them as function values, or even storing them in lists or other containers, etc), and there's no namespace distinction between functions and other kinds of objects, no automatic calling of functions just because they're mentioned, etc, etc. (It's harder - a bit harder, or MUCH harder, depending - in other languages that do draw lots of distinctions of this sort). In Python, mentioning a function is just that -- a mention; the CALL only happens if and when the function object (referred to by name, or otherwise) is followed by parentheses.

That's about all there is to this example -- please do feel free to edit your question, comment here, etc, if there's some other specific aspect that you remain in doubt about!

Edit: so the OP commented courteously asking for more examples of "closure factories". Here's one -- imagine some abstract kind of GUI toolkit, and you're trying to do:

for i in range(len(buttons)):
  buttons[i].onclick(lambda: mainwin.settitle("button %d click!" % i))

but this doesn't work right -- i within the lambda is late-bound, so by the time one button is clicked i's value is always going to be the index of the last button, no matter which one was clicked. There are various feasible solutions, but a closure factory's an elegant possibility:

def makeOnclick(message):
  return lambda: mainwin.settitle(message)

for i in range(len(buttons)):
  buttons[i].onclick(makeOnClick("button %d click!" % i))

Here, we're using the closure factory to tweak the binding time of variables!-) In one specific form or another, this is a pretty common use case for closure factories.