Dynamic module import in Python

Question

I'm writing a Python application that takes as a command as an argument, for example:

$ python myapp.py command1

I want the application to be extensible, that is, to be able to add new modules that implement new commands without having to change the main application source. The tree looks something like:

myapp/
    __init__.py
    commands/
        __init__.py
        command1.py
        command2.py
    foo.py
    bar.py

So I want the application to find the available command modules at runtime and execute the appropriate one.

Currently this is implemented something like:

command = sys.argv[1]
try:
    command_module = __import__("myapp.commands.%s" % command, fromlist=["myapp.commands"])
except ImportError:
    # Display error message

command_module.run()

This works just fine, I'm just wondering if there is possibly a more idiomatic way to accomplish what we are doing with this code.

Note that I specifically don't want to get in to using eggs or extension points. This is not an open-source project and I don't expect there to be "plugins". The point is to simplify the main application code and remove the need to modify it each time a new command module is added.

Answer 1

You can use exec:

exec "import myapp.commands.%s" % command

Answer 2

Nope, that's pretty much how to do it. You can use exec if you want to as well.

Note you can import a list of modules by doing this:

>>> moduleNames = ['sys', 'os', 're', 'unittest'] 
>>> moduleNames
['sys', 'os', 're', 'unittest']
>>> modules = map(__import__, moduleNames)

Ripped straight from Dive Into Python.

Answer 3

Use the imp module, or the more direct __import__() function.

Answer 4

As mentioned the imp module provides you loading functions.

imp.load_source(path)

imp.load_compiled(path)

I've used these before to perform something similar.
In my case I defined a specific class with defined methods that were required. So, once I loaded the module I would check if the class was in the module, and then create an instance of that class.

Something like this:

import imp


def load_from_file(filepath):
    class_inst = None
    expected_class = 'MyClass'

    mod_name,file_ext = os.path.splitext(os.path.split(filepath)[-1])

    if file_ext.lower() == '.py':
        py_mod = imp.load_source(mod_name, filepath)

    elif file_ext.lower() == '.pyc'
        py_mod = imp.load_compiled(mod_name, filepath)

    if expected_class in dir(py_mod):
        class_inst = py_mod.MyClass() 

    return class_inst

Answer 5

It sounds like what you really want is a plugin architecture.

You should have a look at the entry points functionality provided by the setuptools package. It offers a great way to discover plugins that are loaded for your application.

Answer 6

Check out this post on importing module code dynamically

Dynamically executing or importing python code

Answer 7

The following worked for me:

import sys, glob
sys.path.append('/home/marc/python/importtest/modus')
fl = glob.glob('modus/*.py')
modulist = []
adapters=[]
for i in range(len(fl)):
    fl[i] = fl[i].split('/')[1]
    fl[i] = fl[i][0:(len(fl[i])-3)]
    modulist.append(getattr(__import__(fl[i]),fl[i]))
    adapters.append(modulist[i]())

It loads modules from the folder 'modus'. The modules have a single class with the same name as the module name. E.g. the file modus/modu1.py contains:

class modu1():
    def __init__(self):
        self.x=1
        print self.x

The result is a list of dynamically loaded classes "adapters".