Python's __import__ doesn't work as expected

Question

When using __import__ with a dotted name, something like: somepackage.somemodule, the module returned isn't somemodule, whatever is returned seems to be mostly empty! what's going on here?

Answer 1

From the python docs on __import__:

__import__( name[, globals[, locals[, fromlist[, level]]]])

...

When the name variable is of the form package.module, normally, the top-level package (the name up till the first dot) is returned, not the module named by name. However, when a non-empty fromlist argument is given, the module named by name is returned. This is done for compatibility with the bytecode generated for the different kinds of import statement; when using "import spam.ham.eggs", the top-level package spam must be placed in the importing namespace, but when using "from spam.ham import eggs", the spam.ham subpackage must be used to find the eggs variable. As a workaround for this behavior, use getattr() to extract the desired components. For example, you could define the following helper:

def my_import(name):
    mod = __import__(name)
    components = name.split('.')
    for comp in components[1:]:
        mod = getattr(mod, comp)
    return mod

To paraphrase:

When you ask for somepackage.somemodule, __import__ returns somepackage.__init__.py, which is often empty.

It will return somemodule if you provide fromlist (a list of the variable names inside somemodule you want, which are not actually returned)

You can also, as I did, use the function they suggest.

Note: I asked this question fully intending to answer it myself. There was a big bug in my code, and having misdiagnosed it, it took me a long time to figure it out, so I figured I'd help the SO community out and post the gotcha I ran into here.

Answer 2

There is something that works as you want it to: twisted.python.reflect.namedAny:

>>> from twisted.python.reflect import namedAny
>>> namedAny("operator.eq")
<built-in function eq>
>>> namedAny("pysqlite2.dbapi2.connect")
<built-in function connect>
>>> namedAny("os")
<module 'os' from '/usr/lib/python2.5/os.pyc'>