How can I deal with python eggs for multiple platforms in one location?

Question

We have a common python installation for all of our systems in order to ensure every system has the same python installation and to ease configuration issues. This installation is located on a shared drive. We also have multiple platforms that share this installation. We get around conflicting platform-specific files by setting the --exec-prefix configure option when compiling python.

My issue is that I now want to install an egg using easy_install (or otherwise) that is platform-dependent. easy_install puts the egg in the site-packages directory of the platform-independent part of the install. The name of the egg has the platform in it so there should be no conflict. But python will only load the first one it finds. (So, on Solaris it might try to load the Linux egg). Modifying the easy-install.pth file can change which one it finds, but that's pretty useless.

I can move the .egg files into a platform-depended packages directory and then use pkg_resources.require() to load them (or manually adjust the path). But it seems as though I shouldn't have to since the platform is in the name of the egg.

Is there any more generic way I can ensure that python will load the egg for the correct platform?

Answer 1

Python has enough hooks to make this possible but it is probably not wise. If you really want to implement this, look at PEP 302 and start hacking on the easy_install or distribute source code. As you know, there's not really any logic in easy-install.pth.

You would probably be much better off simply keeping track of which Python packages you want installed. You could for example write your own Python package that just depends on the desired set.

Answer 2

Try virtualenv ... http://pypi.python.org/pypi/virtualenv ... helps you create isolated environment with it's own python interpreter + site_packages folder. Thus you never have any conflicts with packages installed in say local path.

Answer 3

What I ended up going with was manually moving the platform-dependent egg to the platform-specific site-packages directory (as specified at http://docs.python.org/install/index.html). Then I made another easy-install.pth in that same directory, listing the eggs to be installed.

This would be much more convenient if easy_install were to honour the exec_prefix and put platform-dependent eggs in the correct "non-pure module distribution" location. Perhaps I will request this from the easy_install folks.

Answer 4

Use "easy_install -m" to install all the platform-specific packages, so that there is no default version on sys.path. That way, version resolution takes place at runtime, and platform information will be taken into consideration.