Weak reference callback is not called because of circular references

Question

I'm trying to write a finalizer for Python classes that have circular references. I found out that weak reference callbacks are the way to go. Unfortunately, it seems the lambda I use as a callback is never called. For example, running this code:

def del_A(name):
    print('An A deleted:' + name)

class A(object):
    def __init__(self, name):
        print('A created')
        self.name = name
        self._wr = weakref.ref(self, lambda wr, n = self.name: del_A(n))

class B(object):
    def __init__(self):
        print('B created')

if __name__ == '__main__':
    a = A('a1')
    b = B()
    a.other = b
    b.other = a

returns:

A created
B created

Removing the circular reference makes the lambda callback works ('An A deleted: a1' is printed). Replacing the lambda by a simple function call works too, but the parameter value is fixed when initializing the weak reference, and not when calling the callback:

self._wr = weakref.ref(self, del_A(self.name))
...
a = A('a1')
a.name = 'a2'
b = B()
a.other = b
b.other = a

returns:

A created
An A deleted:a1
B created

Any idea why the lambda callback does not work with circular references?

Answer 1

Circular references are cleaned up automatically. There are a few exceptions, such as classes that define a __del__ method.

Usually you do not need to define a __del__ method

Answer 2

When you use

 self._wr = weakref.ref(self, lambda wr, n = self.name: del_A(n))  

the callback will only be called when self is about to be finalized.

The reason why the callback is not getting called is because

a = A('a1')
b = B()
a.other = b   # This gives a another attribute; it does not switch `a` away from the original `a`
b.other = a

does not cause a to be finalized. The original a still exists.

The callback would be called if you changed the code to

a = A('a1')
b = B()
a = b
b = a

When you use

self._wr = weakref.ref(self, del_A(self.name))

then your callback is None. del_A(self.name) is not a reference to a function, it is a function call itself. So del_A(self.name) prints An A deleted:a1 immediately (before a1 is really finalized), and returns with the value None, which becomes the default callback for the weakref.

Answer 3

I think I finally found the reason why the callback is not called in the presence of a weak reference:

Weak reference callbacks are not called if the "weakref object dies before the object it references"

It seems that when circular references are deleted, the weak reference attribute of class A is deleted before the callback has a chance to be called. One solution, is to append the finalizer (i.e., the weak reference and its callback) to a list of finalizers. For example:

def del_A(name):
    print('An A deleted:' + name)

class A(object):
    def __init__(self, name, finalizers):
        print('A created')
        self.name = name
        finalizers.append(weakref.ref(self, lambda wr, n = self.name: del_A(n)))

class B(object):
    def __init__(self):
        print('B created')

def do_work(finalizers):
    a = A('a1', finalizers)
    b = B()
    a.other = b
    b.other = a

if __name__ == '__main__':
    finalizers = []
    do_work(finalizers)

will print:

A created
B created
An A deleted:a1

Note that do_work() is necessary, otherwise finalizers gets deleted before the callbacks have a chance to be called. Obviously, finalizers has to be managed properly to avoid building a huge list of weak references, but this is another issue.