Python weakref callbacks and __del__ execution order

Question

In Python, is there a way to call a function after an object is finalized?

I thought the callback in a weakref would do it, but it appears a weakref's callback is called once the object is garbage collected, but before the objects __del__ method is called. This seems contrary to the notes on weakrefs and garbace collection in the Python trunk. Here's an example.

import sys
import weakref

class Spam(object) :
  def __init__(self, name) :
    self.name = name

  def __del__(self) :
    sys.stdout.write("Deleting Spam:%s\n" % self.name)
    sys.stdout.flush()

def cleaner(reference) :
  sys.stdout.write("In callback with reference %s\n" % reference)
  sys.stdout.flush()

spam = Spam("first")
wk_spam = weakref.ref(spam, cleaner)
del spam

The output I get is

$ python weakref_test.py 
In callback with reference <weakref at 0xc760a8; dead>
Deleting Spam:first

Is there some other conventional way to do what I want? Can I somehow force the finalization in my callback?

Answer 1

If "do what you want" means "run code when a resource leaves the context" ( instead of, for example, "abuse the garbage collector to do stuff" ), you're looking in the wrong direction. Python abtracted that whole idea as context-managers, used with the with statement.

from __future__ import with_statement
import sys
class Spam(object):
    def __init__(self, name):
        self.name = name

    def __enter__(self):
        sys.stdout.write("Entering Spam:%s\n" % self.name)
        sys.stdout.flush()

    def __exit__(self, type, value, traceback):
        sys.stdout.write("Lets clean up Spam:%s\n" % self.name) 
        if type is None:
            sys.stdout.write("Leaving Spam:%s in peace\n" % self.name)
            return
        else:
            sys.stdout.write("Leaving Spam:%s with Exception:%r\n" % (self.name, value))


with Spam("first") as spam:
    pass

with Spam("2nd") as spam:
    raise Exception("Oh No!")

gives:

Entering Spam:first
Lets clean up Spam:first
Leaving Spam:first in peace
Entering Spam:2nd
Lets clean up Spam:2nd
Leaving Spam:2nd with Exception:Exception('Oh No!',)
Traceback (most recent call last):
  File "asd.py", line 24, in <module>
    raise Exception("Oh No!")
Exception: Oh No!

Answer 2

Here is a solution that uses a serialized garbage loop on another thread. Its probably the closest solution you'll get.

import sys
from threading import Thread
from Queue import Queue
import weakref

alive = weakref.WeakValueDictionary()
dump = Queue()

def garbageloop():
    while True:
        f = dump.get()
        f()

garbage_thread = Thread(target=garbageloop)
garbage_thread.daemon = True
garbage_thread.start()

class Spam(object) :
  def __init__(self, name) :
    self.name = name
    alive[id(self)] = self

  def __del__(self) :
    sys.stdout.write("Deleting Spam:%s\n" % self.name)
    sys.stdout.flush()
    dump.put(lambda:cleaner(id(self)))

def cleaner(address):
  if address in alive:
    sys.stdout.write("Object was still alive\n")
  else:
    sys.stdout.write("Object is dead\n")
  sys.stdout.flush()

spam = Spam("first")
del spam