views:

89

answers:

2

I have a directory structure as follows:

| main.py
| scripts
|--| __init__.py
   | script1.py
   | script2.py
   | script3.py

From main.py, the module scripts is imported. I tried using pkgutils.walk_packages in combination with __all__, but using that, I can only import all the submodules directly under main using from scripts import *. I would like to get them all under scripts. What would be the cleanest way to import all the submodules of scripts so that I could access scripts.script1 from main?

EDIT: I am sorry that I was a bit vague. I would like to import the submodules on run-time without specifying them explicitly in __init__.py. I can use pkgutils.walk_packages to get the submodule names (unless someone knows of a better way), but I am not sure of the cleanest way to use these names (or maybe the ImpImporters that walk_packages returns?) to import them.

+3  A: 

Edit: Here's one way to recursively import everything at runtime...

It uses exec, so there's almost certainly a better way, but it does work (even for arbitrarily nested sub-packages, I think).

(Contents of __init__.py in top package directory)

import pkgutil

__all__ = []
for loader, module_name, is_pkg in  pkgutil.walk_packages(__path__):
    __all__.append(module_name)
    module = loader.find_module(module_name).load_module(module_name)
    exec('%s = module' % module_name)

I'm not using __import__(__path__+'.'+module_name) here, as it's difficult to properly recursively import packages using it. If you don't have nested sub-packages, and wanted to avoid the exec/eval, though, it's one way to do it.

There's probably a better way, but this is the best I can do, anyway.

Original Answer (For context, ignore othwerwise. I misunderstood the question initially):

What does your scripts/__init__.py look like? It should be something like:

import script1
import script2
import script3
__all__ = ['script1', 'script2', 'script3']

You could even do without defining __all__, but things (pydoc, if nothing else) will work more cleanly if you define it, even if it's just a list of what you imported.

Joe Kington
Sorry for not explaining fully. I have edited the main post. I am looking to import the submodules dynamically at run-time.
linkmaster03
@linkmaster03 - Sorry for the misunderstanding. See the changes above... It's ugly, but it works. There's probably a better way, though.
Joe Kington
Thanks! :) I found that using `__import__` and not needing to import the scripts by their names actually worked out nicely, because I could just iterate over the modules without needing their names.
linkmaster03
+3  A: 

I was writing a small personal library and adding new modules all the time so I wrote a shell script to look for scripts and create the __init__.py's. The script is executed just outside of the main directory for my package, pylux.

I know it probably isn't the answer you're looking for, but it servered its purpose for me and it might be useful to someone else, too.

#!/bin/bash

echo 'Traversing folder hierarchy...'

CWD=`pwd`


for directory in `find pylux -type d -exec echo {} \;`;
do
    cd $directory
    #echo Entering $directory
    echo -n "" > __init__.py

    for subdirectory in `find . -type d -maxdepth 1 -mindepth 1`;
    do
        subdirectory=`echo $subdirectory | cut -b 3-`
        #echo -n '    ' ...$subdirectory
        #echo -e '\t->\t' import $subdirectory
        echo import $subdirectory >> __init__.py
    done

    for pyfile in *.py ;
    do
        if [ $pyfile = $(echo __init__.py) ]; then
            continue
        fi
        #echo -n '    ' ...$pyfile
        #echo -e '\t->\t' import `echo $pyfile | cut -d . -f 1`
        echo import `echo $pyfile | cut -d . -f 1` >> __init__.py
    done
    cd $CWD

done


for directory in `find pylux -type d -exec echo {} \;`;
do
    echo $directory/__init__.py:
    cat $directory/__init__.py | awk '{ print "\t"$0 }'
done
vgm64
Couldn't the same logic be written in `__init__.py`? Using `os.listdir` and `__import__`?
cji
Hmm. I suppose so, but I'd have to look into it a bit more.
vgm64