How do I make Python pick the correct module without manually modifying sys.path?

Question

Hi,

I have made some changes in a python module in my checked out copy of a repository, and need to test them. However, when I try to run a script that uses the module, it keeps importing the module from the trunk of the repository, which is of no use to me.

I tried setting PYTHONPATH, which did nothing at all. After some searching around, I found that anything in the .pth files under site-packages directory will be put in even before PYTHONPATH (which to me defeats the purpose of having it). I believe this is the cause for my module not being picked.

Am I correct? If so, what is the way to override this (without modifying the script to have a sys.path.insert(0,path) )?

Edit: In reply to NicDumz - the original repository was under /projects/spam. The python modules were part of this in /projects/spam/sources/python/a/b/. However, these are 'built' every night using a homegrown make variant which then puts them into /projects/spam/build/lib/python/a/b/. The script is using the module under this last path only.
I have checked out the entire repository to under /home/sundar/spam, and made changes in /home/sundar/spam/sources/python/a/b/mymodule.py. I've set my PYTHONPATH to /home/sundar/spam/sources/python and tried to import a.b.mymodule with no success.

Answer 1

You could write a small script, such as the one below, that prefixes sys.path, then set PYTHONSTARTUP to use that script.

import sys
sys.path.insert(0, 'c:/temp')

For example...

C:\temp>set PYTHONSTARTUP=c:\temp\tst.py
C:\temp>C:\Python26\python
Python 2.6.2 (r262:71605, Apr 14 2009, 22:40:02) [MSC v.1500 32 bit (Intel)] on
win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> sys.path
['c:/temp', '', 'C:\\Python26\\lib\\site-packages\\setuptools-0.6c9-py2.6.egg',
'C:\\Python26\\lib\\site-packages\\pyyaml-3.08-py2.6-win32.egg', 'C:\\Python26\\
lib\\site-packages\\pyglet-1.1.3-py2.6.egg', 'C:\\Python26\\lib\\site-packages\\
simpy-2.0.1-py2.6.egg', 'C:\\Python26\\lib\\site-packages\\nose-0.11.1-py2.6.egg
', 'C:\\Python26\\lib\\site-packages\\mercurial-unknown-py2.6-win32.egg', 'c:\\t
emp', 'C:\\WINDOWS\\system32\\python26.zip', 'C:\\Python26\\DLLs', 'C:\\Python26
\\lib', 'C:\\Python26\\lib\\plat-win', 'C:\\Python26\\lib\\lib-tk', 'C:\\Python2
6', 'C:\\Python26\\lib\\site-packages', 'C:\\Python26\\lib\\site-packages\\win32
', 'C:\\Python26\\lib\\site-packages\\win32\\lib', 'C:\\Python26\\lib\\site-pack
ages\\Pythonwin']

Answer 2

It sounds like you need to install virtualenv and use it to set up different environments for different purposes. In one environment, you would import modules from the trunk of the repository, but in another environment you would have a mixture of trunk modules and test modules.

By keeping everything separate like this you make it easier to rollback changes (just delete the whole virtual environment folder) and you greatly reduce the risk that your test rigging will end up being committed to the repository.

Answer 3

You can create a setup script with setuptools or distribute, then do a python setup.py develop. It will add a link to your working copy in a .pth file, overriding any installed version.

When you are done, you can simply delete the link in the .pth file.

Answer 4

Your current working directory is first in the sys.path. Anything there trumps anything else on the path.

Copy the "test version" to some place closer to the front of the list of directories in sys.path, like your current working directory.