Ubuntu + virtualenv = a mess? virtualenv hates dist-packages, wants site-packages

Question

Can someone please explain to me what is going on with python in ubuntu 9.04?

I'm trying to spin up virtualenv, and the --no-site-packages flag seems to do nothing with ubuntu. I installed virtualenv 1.3.3 with easy_install (which I've upgraded to setuptools 0.6c9) and everything seems to be installed to /usr/local/lib/python2.6/dist-packages

I assume that when installing a package using apt-get, it's placed in /usr/lib/python2.6/dist-packages/ ?

The issue is, there is a /usr/local/lib/python2.6/site-packages as well that just sits there being empty. It would seem (by looking at the path in a virtualenv) that this is the folder virtualenv uses as backup. Thus even thought I omit --no-site-packages, I cant access my local systems packages from any of my virtualenv's.

So my questions are:

1. How do I get virtualenv to point to one of the dist-packages?
2. Which dist-packages should I point it to? /usr/lib/python2.6/dist-packages or /usr/local/lib/python2.6/dist-packages/
3. What is the point of /usr/lib/python2.6/site-packages? There's nothing in there!
4. Is it first come first serve on the path? If I have a newer version of package XYZ installed in /usr/local/lib/python2.6/dist-packages/ and and older one (from ubuntu repos/apt-get) in /usr/lib/python2.6/dist-packages, which one gets imported when I import xyz? I'm assuming this is based on the path list, yes?
5. Why the hell is this so confusing? Is there something I'm missing here?
6. Where is it defined that easy_install should install to /usr/local/lib/python2.6/dist-packages?
7. Will this affect pip as well?

Thanks to anyone who can clear this up!

Answer 1

I'd be tempted to hack it by making site-packages a link to dist-packages, but I guess this might affect other cases where you want to install some extension other than from the ubuntu dist. I can't think of another answer to 1 except tweaking virtualenv's sources (with both ubuntu and virtualenv being so popular I wouldn't be surprised to find tweaked versions already exist).

Re 2, if you're using /usr/local/bin/python you should use the /usr/local version of the lib (including site-packages) and conversely if you're using /usr/bin/python.

Re 3, there will be something there if you ever install an extension for /usr/bin/python from sources (not via easy_install or from ubuntu's distro).

Re 4, yes, earlier entries on the path take precedence.

Re 5, easy_install is easy only in its name -- it does so much dark magic that it's been carefully kept out of the standard python library despite its convenience because the consensus among us python committers is that deep dark magic for convenience is "easy" only on the surface.

Re 6, I think that's an ubuntu modification to easy_install -- if that's right then it's defined wherever Canonical or other ubuntu maintainers make their collective decisions.

Re 7, sorry, no idea -- I have no reasonably recent ubuntu at hand to check.

Answer 2

Well I have a Ubuntu 9.04 and quickly tried setting up a couple sandboxes with site-packages and one without. And things are working fine.

The only difference in the approach I took is I used Ubuntu's python-virtualenv package (1.3.3). And presume that it is tweaked by Ubuntu team to suit Ubuntu setups.

To sum up disable easy_installed virtualenv for a while, use packaged python-virtualenv and see if that meets your expectations.

In fact we use similar setup for production without any problem. Rest is already answered by Alex.

Answer 3

I believe Mike Orr's answer from the virtual-env mailing list seems to be the best. Note the OP published this question in both places.