How to monitor Python files for changes?

Question

I want to restart my Python web application, if code gets changed. But there could be a large number of files that could be changed, since files in imported modules could change ...

How to get the actual file names from imported packages / modules?

How can modified Python files be detected efficiently? Is there a library to do that?

Answer 1

This is operating system specific. For Linux, there is inotify, see e.g. http://github.com/rvoicilas/inotify-tools/

Answer 2

gamin is another option which is slightly less Linux-specific.

Answer 3

I'm not sure how you would implement the 'reload application' operation in your circumstance; reloading a changed module with the reload built-in probably won't cut it.

But as far as detecting whether or not there was a change, the following would be one way to approach it.

• Most python modules have a __file__ attribute.
• All loaded modules are stored in sys.modules.
• We can walk through sys.modules at some interval, and look for changes on disk for each module in turn

Sometimes __file__ points to a .pyc file instead of a .py file, so you might have to chop off the trailing c. Sometimes a .pyc file exists but a .py doesn't exist; in a robust system you'd have to allow for this.

A proof of concept of code to do this (not robust):

_module_timestamps = {}
_checking = False

def run_checker():
    global _checking
    _checking = True
    while _checking:
        for name, module in sys.modules.iteritems():
            if hasattr(module, '__file__'):
                filename = module.__file__
                if filename.endswith('.pyc'):
                    filename = filename[:-1]
                mtime = os.stat(filename).st_mtime
                if name not in _module_timestamps:
                    _module_timestamps[name] = mtime
                else:
                    if mtime > _module_timestamps[name]:
                        do_reload(name)
            else:
                'module %r has no file attribute' % (name,)
        time.sleep(1)

def do_reload(modname):
    print 'I would reload now, because of %r' % (modname,)

check_thread = threading.Thread(target=run_checker)
check_thread.daemon = True
check_thread.start()

try:
    while 1:
        time.sleep(0.1)
except KeyboardInterrupt:
    print '\nexiting...'