Setting Python path while developing library module

Question

I am developing a library and an application that uses the library in Python 2.6. I've placed a "mylib.pth" file in "site-packages" so that I can import mylib from within my application.

I am using a DVCS so when I want to fix a bug or add a feature to the library I make a branch of the repository and work within that branch. To test my application with the changes I am making to the library I edit the path in "mylib.pth" to point to the new development branch.

This gets a little tedious if I have a few parallel branches of my library going on at one. I have to keep editing the "mylib.pth" file before testing to ensure I am testing against the correct version of my library. Is there a way to use the current path (i.e. the development branch of the library that I am current in) to set the library path when I invoke my application instead of using the "mylib.pth" in the global "site-packages" directory?

Answer 1

Is virtualenv what you're looking for? From the description:

Imagine you have an application that needs version 1 of LibFoo, but another application requires version 2. How can you use both these applications? If you install everything into /usr/lib/python2.4/site-packages (or whatever your platform's standard location is), it's easy to end up in a situation where you unintentionally upgrade an application that shouldn't be upgraded.

Answer 2

Suggested reading: Tools of the Modern Python Hacker: Virtualenv, Fabric and Pip. It addresses a number of problems with development and deployment of Python apps.

Answer 3

If you use setuptools, then you can say setup.py develop in your working tree, and it will do the .pth file manipulation for you.

Answer 4

Sure, you can alter sys.path to add the current directory (or a subdirectory of it) to the search path. site.addsitedir is a good way to do it. Since you'd be doing this from Python you can have any sort of logic you like for deciding which directory to add; you could base it on os.path.normpath​ing the current directory if it looks like a branch, or looking for the newest branch on-disc, or something else.

You could put this code in the sitecustomize.py module or other startup-triggered location.

Answer 5

You might also consider using zc.buildout. It allows you to create entry points with customized python paths.

Answer 6

I set my PYTHONPATH to point to the latest-and-greatest version. No editing.

export PYTHONPATH=.:/the/new/version