How to connect to a GObject signal in python, without it keeping a reference to the connecter?

Question

The problem is basically this, in python's gobject and gtk bindings. Assume we have a class that binds to a signal when constructed:

class ClipboardMonitor (object):
  def __init__(self):
    clip = gtk.clipboard_get(gtk.gdk.SELECTION_CLIPBOARD)
    clip.connect("owner-change", self._clipboard_changed)

The problem is now that, no instance of ClipboardMonitor will ever die. The gtk clipboard is an application-wide object, and connecting to it keeps a reference to the object, since we use the callback self._clipboard_changed.

I'm debating how to work around this using weak references (weakref module), but I have yet to come up with a plan. Anyone have an idea how to pass a callback to the signal registration, and have it behave like a weak reference (if the signal callback is called when the ClipboardMonitor instance is out of scope, it should be a no-op).

Addition: Phrased independently of GObject or GTK+:

How do you provide a callback method to an opaque object, with weakref semantics? If the connecting object goes out of scope, it should be deleted and the callback should act as a no-op; the connectee should not hold a reference to the connector.

To clarify: I explicitly want to avoid having to call a "destructor/finalizer" method

Answer 1

The standard way is to disconnect the signal. This however needs to have a destructor-like method in your class, called explicitly by code which maintains your object. This is necessary, because otherwise you'll get circular dependency.

class ClipboardMonitor(object):
    [...]

    def __init__(self):
        self.clip = gtk.clipboard_get(gtk.gdk.SELECTION_CLIPBOARD)
        self.signal_id = self.clip.connect("owner-change", self._clipboard_changed)

    def close(self):
        self.clip.disconnect(self.signal_id)

As you pointed out, you need weakrefs if you want to avoid explicite destroying. I would write a weak callback factory, like:

import weakref

class CallbackWrapper(object):
    def __init__(self, sender, callback):
        self.weak_obj = weakref.ref(callback.im_self)
        self.weak_fun = weakref.ref(callback.im_func)
        self.sender = sender
        self.handle = None

    def __call__(self, *things):
        obj = self.weak_obj()
        fun = self.weak_fun()
        if obj is not None and fun is not None:
            return fun(obj, *things)
        elif self.handle is not None:
            self.sender.disconnect(self.handle)
            self.handle = None
            self.sender = None

def weak_connect(sender, signal, callback):
    wrapper = CallbackWrapper(sender, callback)
    wrapper.handle = sender.connect(signal, wrapper)
    return wrapper

(this is a proof of concept code, works for me -- you should probably adapt this piece to your needs). Few notes:

• I am storing callback object and function separatelly. You cannot simply make a weakref of a bound method, because bound methods are very temporary objects. Actually weakref.ref(obj.method) will destroy the bound method object instantly after creating a weakref. I didn't checked it it is needed to store a weakref to the function too... I guess if your code is static, you probably can avoid that.
• The object wrapper will remove itself from the signal sender when it notices that the weak reference ceased to exist. This is also necessary to destroy the circular dependence between the CallbackWrapper and the signal sender object.

Answer 2

(This answer tracks my progress)

This second version will disconnect as well; I have a convenience function for gobjects, but I actually need this class for a more general case -- both for D-Bus signal callbacks and GObject callbacks.

Anyway, what can one call the WeakCallback implementation style? It is a very clean encapsulation of the weak callback, but with the gobject/dbus specialization unnoticeably tacked on. Beats writing two subclasses for those two cases.

import weakref

class WeakCallback (object):
    """A Weak Callback object that will keep a reference to
    the connecting object with weakref semantics.

    This allows to connect to gobject signals without it keeping
    the connecting object alive forever.

    Will use @gobject_token or @dbus_token if set as follows:
        sender.disconnect(gobject_token)
        dbus_token.remove()
    """
    def __init__(self, obj, attr):
        """Create a new Weak Callback calling the method @obj.@attr"""
        self.wref = weakref.ref(obj)
        self.callback_attr = attr
        self.gobject_token = None
        self.dbus_token = None

    def __call__(self, *args, **kwargs):
        obj = self.wref()
        if obj:
            attr = getattr(obj, self.callback_attr)
            attr(*args, **kwargs)
        elif self.gobject_token:
            sender = args[0]
            sender.disconnect(self.gobject_token)
            self.gobject_token = None
        elif self.dbus_token:
            self.dbus_token.remove()
            self.dbus_token = None

def gobject_connect_weakly(sender, signal, connector, attr, *user_args):
    """Connect weakly to GObject @sender's @signal,
    with a callback in @connector named @attr.
    """
    wc = WeakCallback(connector, attr)
    wc.gobject_token = sender.connect(signal, wc, *user_args)

Answer 3

not actually tried it yet, but:

class WeakCallback(object):
    """
    Used to wrap bound methods without keeping a ref to the underlying object.
    You can also pass in user_data and user_kwargs in the same way as with
    rpartial. Note that refs will be kept to everything you pass in other than
    the callback, which will have a weakref kept to it.
    """
    def __init__(self, callback, *user_data, **user_kwargs):
        self.im_self = weakref.proxy(im_self, self._invalidated)
        self.im_func = weakref.proxy(im_func)
        self.user_data = user_data
        self.user_kwargs = user_kwargs

    def __call__(self, *args, **kwargs):
        kwargs.update(self.user_kwargs)
        args += self.user_data
        self.im_func(self.im_self, *args, **kwargs)

    def _invalidated(self, im_self):
        """Called by the weakref.proxy object."""
        cb = getattr(self, 'cancel_callback', None)
        if cb is not None:
            cb()

    def add_cancel_funtion(self, cancel_callback):
        """
        A ref will be kept to cancel_callback. It will be called back without
        any args when the underlying object dies.
        You can wrap it in WeakCallback if you want, but that's a bit too
        self-referrential for me to do by default. Also, that would stop you
        being able to use a lambda as the cancel_callback.
        """
        self.cancel_callback = cancel_callback

def weak_connect(sender, signal, callback):
    """
    API-compatible with the function described in
    http://stackoverflow.com/questions/1364923/. Mostly used as an example.
    """
    cb = WeakCallback(callback)
    handle = sender.connect(signal, cb)
    cb.add_cancel_function(WeakCallback(sender.disconnect, handle))