Numeric variable scope in Python closure (Python v2.5.2)

Question

I have a nested function where I am trying to access variables assigned in the parent scope. From the first line of the next() function I can see that path, and nodes_done are assigned as expected. distance, current, and probability_left have no value and are causing a NameError to be thrown.

What am I doing wrong here? How can I access and modify the values of current, distance, and probability_left from the next() function?

def cheapest_path(self):
    path = []
    current = 0
    distance = 0
    nodes_done = [False for _ in range(len(self.graph.nodes))]
    probability_left = sum(self.graph.probabilities)

    def next(dest):
        log('next: %s -> %s distance(%.2f), nodes_done(%s), probability_left(%.2f)' % (distance,self.graph.nodes[current],self.graph.nodes[dest],str(nodes_done),probability_left))
        path.append((current, distance, nodes_done, probability_left))

        probability_left -= self.graph.probabilities[current]
        nodes_done[current] = True
        distance = self.graph.shortest_path[current][dest]
        current = dest

    def back():
        current,nodes_done,probability_left = path.pop()

Answer 1

The way Python's nested scopes work, you can never assign to a variable in the parent scope, unless it's global (via the global keyword). This changes in Python 3 (with the addition of nonlocal), but with 2.x you're stuck.

Instead, you have to sort of work around this by using a datatype which is stored by reference:

def A():
    foo = [1]
    def B():
        foo[0] = 2 # since foo is a list, modifying it here modifies the referenced list

Note that this is why your list variables work - they're stored by reference, and thus modifying the list modifies the original referenced list. If you tried to do something like path = [] inside your nested function, it wouldn't work because that would be actually assigning to path (which Python would interpret as creating a new local variable path inside the nested function that shadows the parent's path).

One option that is sometimes used is to just keep all of the things that you want to persist down into the nested scope in a dict:

def A():
    state = {
        'path': [],
        'current': 0,
        # ...
    }

    def B():
        state['current'] = 3

Answer 2

The short answer is that python does not have proper lexical scoping support. If it did, there would have to be more syntax to support the behavior (i.e. a var/def/my keyword to declare the variable scope).

Barring actual lexical scoping, the best you can do is store the data in an environment data structure. One simple example would be a list, e.g.:

def cheapest_path(self):
    path = []
    path_info = [0, 0]
    nodes_done = [False for _ in range(len(self.graph.nodes))]
    probability_left = sum(self.graph.probabilities)

    def next(dest):
        distance, current = path_info
        log('next: %s -> %s distance(%.2f), nodes_done(%s), probability_left(%.2f)' %     (distance,self.graph.nodes[current],self.graph.nodes[dest],str(nodes_done),probability_left))
        path.append((current, distance, nodes_done, probability_left))

        probability_left -= self.graph.probabilities[current]
        nodes_done[current] = True
        path_info[0] = self.graph.shortest_path[current][dest]
        path_info[1] = dest

    def back():
        current,nodes_done,probability_left = path.pop()

You can do this or do inspect magic. For more history on this read this thread.

Answer 3

If you happen to be working with Python 3, you can use the nonlocal statement (documentation) to make those variables exist in the current scope, e.g.:

def next(dest):
    nonlocal distance, current, probability_left
    ...