Importing Python modules from different working directory

Question

I have a Python script that uses built-in modules but also imports a number of custom modules that exist in the same directory as the main script itself.

For example, I would call

python agent.py

and agent.py has a number of imports, including:

import checks

where checks is in a file in the same directory as agent.py

agent/agent.py
agent/checks.py

When the current working directory is agent/ then everything is fine. However, if I call agent.py from any other directory, it is obviously unable to import checks.py and so errors.

How can I ensure that the custom modules can be imported regardless of where the agent.py is called from e.g.

python /home/bob/scripts/agent/agent.py

Answer 1

You need to add the path to the currently executing module to the sys.path variable. Since you called it on the command line, the path to the script will always be in sys.argv[0].

import sys
import os
sys.path.append(os.path.split(sys.argv[0])[0])

Now when import searches for the module, it will also look in the folder that hosts the agent.py file.

Answer 2

There are several ways to add things to the PYTHONPATH.

Read http://docs.python.org/library/site.html

1. Set the PYTHONPATH environment variable prior to running your script.

   You can do this python -m agent to run agent.py from your PYTHONPATH.

2. Create .pth files in your lib/site-packages directory.

3. Install your modules in lib/site-packages.

Answer 3

Actually your example works because checks.py is in the same directory as agent.py, but say checks.py was in the preceeding directory, eg;

agent/agent.py
checks.py

Then you could do the following:

path = os.path.abspath(os.path.join(os.path.dirname(__file__), '..'))
if not path in sys.path:
    sys.path.insert(1, path)
del path

Note the use of __file__.

Answer 4

You should NOT need to fiddle with sys.path. To quote from the Python 2.6 docs for sys.path: """ As initialized upon program startup, the first item of this list, path[0], is the directory containing the script that was used to invoke the Python interpreter. If the script directory is not available (e.g. if the interpreter is invoked interactively or if the script is read from standard input), path[0] is the empty string, which directs Python to search modules in the current directory first. Notice that the script directory is inserted before the entries inserted as a result of PYTHONPATH. """

=== amod.py ===
def whoami():
    return __file__

=== ascript.py ===
import sys
print "sys.argv", sys.argv
print "sys.path", sys.path
import amod
print "amod __file__", amod.whoami()

=== result of running ascript.py from afar ===
C:\somewhere_else>\python26\python \junk\timport\ascript.py
sys.argv ['\\junk\\timport\\ascript.py']
sys.path ['C:\\junk\\timport', 'C:\\WINDOWS\\system32\\python26.zip', SNIP]
amod __file__ C:\junk\timport\amod.py

and if it's re-run, the last line changes of course to ...amod.pyc. This appears not to be a novelty, it works with Python 2.1 and 1.5.2.

Debug hints for you: Try two simple files like I have. Try running Python with -v and -vv. Show us the results of your failing tests, including full traceback and error message, and your two files. Tell us what platform you are running on, and what version of Python. Check the permissions on the checks.py file. Is there a checks.something_else that's causing interference?

Answer 5

I think you should consider making the agent directory into a proper Python package. Then you place this package anywhere on the python path, and you can import checks as

from agent import checks

See http://docs.python.org/tutorial/modules.html

Answer 6

If you know full path to check.py use this recipe (http://code.activestate.com/recipes/159571/)

If you want to add directory to system path -- this recipe (http://code.activestate.com/recipes/52662/). In this case I have to determine application directory (sys.argv[0]) an pass this value to AddSysPath function. If you want to look at production sample please leave a comment on this thread so I post it later.

Regards.