PyGTK/GIO: monitor directory for changes recursively

Question

Take the following demo code (from the GIO answer to this question), which uses a GIO FileMonitor to monitor a directory for changes:

import gio

def directory_changed(monitor, file1, file2, evt_type):
    print "Changed:", file1, file2, evt_type

gfile = gio.File(".")
monitor = gfile.monitor_directory(gio.FILE_MONITOR_NONE, None)
monitor.connect("changed", directory_changed) 

import glib
ml = glib.MainLoop()
ml.run()

After running this code, I can then create and modify child nodes and be notified of the changes. However, this only works for immediate children (I am aware that the docs don't say otherwise). The last of the following shell commands will not result in a notification:

touch one
mkdir two
touch two/three

Is there an easy way to make it recursive? I'd rather not manually code something that looks for directory creation and adds a monitor, removing them on deletion, etc.

The intended use is for a VCS file browser extension, to be able to cache the statuses of files in a working copy and update them individually on changes. So there might by anywhere from tens to thousands (or more) directories to monitor. I'd like to just find the root of the working copy and add the file monitor there.

I know about pyinotify, but I'm avoiding it so that this works under non-Linux kernels such as FreeBSD or... others. As far as I'm aware, the GIO FileMonitor uses inotify underneath where available, and I can understand not emphasising the implementation to maintain some degree of abstraction, but it suggested to me that it should be possible.

(In case it matters, I originally posted this on the PyGTK mailing list.)

Answer 1

"Is there an easy way to make it recursive?"

I'm not aware of any "easy way" to achieve this. The underlying systems, such as inotify on Linux or kqueue on BSDs don't provide facilities to automatically add recursive watches. I'm also not aware of any library layering what you want atop GIO.

So you'll most likely have to build this yourself. As this can be a bit trick in some corner cases (e.g. mkdir -p foo/bar/baz) I would suggest looking at how pynotify implements its auto_add functionality (grep through the pynotify source) and porting that over to GIO.

Answer 2

I'm not sure if GIO allows you to have more than one monitor at once, but if it does there's no* reason you can't do something like this:

import gio
import os

def directory_changed(monitor, file1, file2, evt_type):
    if os.path.isdir(file2):    #maybe this needs to be file1?
        add_monitor(file2) 
    print "Changed:", file1, file2, evt_type

def add_monitor(dir):
    gfile = gio.File(dir)
    monitor = gfile.monitor_directory(gio.FILE_MONITOR_NONE, None)
    monitor.connect("changed", directory_changed) 

add_monitor('.')

import glib
ml = glib.MainLoop()
ml.run()

*when I say no reason, there's the possibility that this could become a resource hog, though with nearly zero knowledge about GIO I couldn't really say. It's also entirely possible to roll your own in Python with a few commands (os.listdir among others). It might look something like this

import time
import os

class Watcher(object):
    def __init__(self):
        self.dirs = []
        self.snapshots = {}

    def add_dir(self, dir):
        self.dirs.append(dir)

    def check_for_changes(self, dir):
        snapshot = self.snapshots.get(dir)
        curstate = os.listdir(dir)
        if not snapshot:
            self.snapshots[dir] = curstate
        else:
            if not snapshot == curstate:
                print 'Changes: ',
                for change in set(curstate).symmetric_difference(set(snapshot)):
                    if os.path.isdir(change):
                        print "isdir"
                        self.add_dir(change)
                    print change,

                self.snapshots[dir] = curstate
                print

    def mainloop(self):
        if len(self.dirs) < 1:
            print "ERROR: Please add a directory with add_dir()"
            return

        while True:
            for dir in self.dirs:
                self.check_for_changes(dir)
            time.sleep(4) # Don't want to be a resource hog

w = Watcher()
w.add_dir('.')


w.mainloop()