Best way to create a "runner" script in Python?

Question

I have a bunch of Python modules in a directory, all being a derivate class. I need a "runner" script that, for each module, instantiate the class that is inside it (the actual class name can be built by the module file name) and than call the "go" method on each of them.

I don't know how many modules are there, but I can list all of them globbing the directory via something like "bot_*.py"

I think this is something about "meta programming", but how could be the best (most elegant) way to do it?

Answer 1

You could use __import__() to load each module, use dir() to find all objects in each module, find all objects which are classes, instantiate them, and run the go() method:

import types
for module_name in list_of_modules_to_load:
    module = __import__(module_name)
    for name in dir(module):
        object = module.__dict__[name]
        if type(object) == types.ClassType:
            object().go()

Answer 2

Here is one way to do this off the top of my head where I have to presume the structure of your modules a bit:

mainDir/
  runner.py
  package/
    __init__.py
    bot_moduleA.py
    bot_moduleB.py
    bot_moduleC.py

In runner you could find this:

import types
import package

for moduleName in dir(package):
  module = package.__dict__[moduleName]
  if type(module) != types.ModuleType:
    continue

  for klassName in dir(module):
    klass = module.__dict__[klassName]
    if type(klass) != types.ClassType:
      continue
    klass().go()

Answer 3

I would try:

import glob
import os

filelist = glob.glob('bot_*.py')
for f in filelist:
    context = {}
    exec(open(f).read(), context)
    klassname = os.path.basename(f)[:-3] 
    klass = context[klassname]
    klass().go()

This will only run classes similarly named to the module, which I think is what you want. It also doesn't have the requirement of the top level directory to be a package.

Beware that glob returns the complete path, including preceding directories, hence the use os.path.basename(f)[:-3] to get the class name.

Answer 4

def run_all(path):
    import glob, os
    print "Exploring %s" % path
    for filename in glob.glob(path + "/*.py"):
        # modulename = "bot_paperino"
        modulename = os.path.splitext(os.path.split(filename)[-1])[0]
        # classname = "Paperino"
        classname = modulename.split("bot_")[-1].capitalize()
        # package = "path.bot_paperino"
        package = filename.replace("\\", "/").replace("/", ".")[:-3]
        mod = __import__(package)
        if classname in mod.__dict__[modulename].__dict__.keys():
            obj = mod.__dict__[modulename].__dict__[classname]()
            if hasattr(obj, "go"):
                obj.go()

if __name__ == "__main__":
    import sys
    # Run on each directory passed on command line
    for path in sys.argv[1:]:
        run_all(sys.argv[1])

You need a __init__.py in each path you want to "run". Change "bot_" at your will. Run on windows and linux.

Answer 5

Great! Thanks to your examples I finally "get it".

I think marcob's snippet is nearly exactly what I'm looking for.

:)