ubuntu9.10 : how to use python's lib-dynload and site-packages directories?

Question

In ubuntu 9.10, in usr/lib/ there are the directories python2.4, python2.5, python2.6 and python3.0

Only python 2.6 is actually working.
python2.4 has only a lib-dynload directory,
python2.5 has only lib-dynload and site-packages,
python3.0 has only a dist-packages directory.

Now i'm wondering what is the idea behind this? Because when i install python2.5 with ./configure, make, make install | altinstall this goes into usr/local/lib and not usr/lib/ so why were these directories added tu ubuntu, how am i supposed to install python to use them?

Answer 1

Sounds like they're an accident from some package(s) you have installed.

The Python version in use determines the locations searched to find installed Python packages/modules, and the "system version" of Python in Ubuntu 9.10 is 2.6, so that's what practically everything should be using. If you were to install the python2.5 package (or it gets installed as a dependency of something else), then it would use /usr/lib/python2.5/*. Try running which python and python --version; also which python2.6 and which python2.5.

From what I understand, though I'm not sure exactly why at all, Debian (from which Ubuntu is derived) uses a dist-packages naming scheme instead of site-packages.

Terminology: Python has packages and Debian (and so Ubuntu) has packages. They aren't the same kind of package, though individual Debian packages will install specific Python packages.

Answer 2

j3ll3, in Ubuntu (or any DPKG-based Linux OS) you can ask the question "What package provides XYZ" by typing

dpkg -S /path/to/XYZ

So, for example, in Ubuntu 9.10,

dpkg -S /usr/lib/python2.5/lib-dynload/gdbm.so

returns

python-gdbm: /usr/lib/python2.5/lib-dynload/gdbm.so

You can find out more about the python-gdbm package by typing

apt-cache show python-gdbm

which says that python-gdbm provides "GNU dbm database support for Python". Perhaps more interestingly, if you type

dpkg --listfiles python-gdbm

you get to see a listing of all the files that python-gdbm installs:

...
/usr/lib/python2.4
/usr/lib/python2.4/lib-dynload
/usr/lib/python2.4/lib-dynload/gdbm.so
/usr/lib/python2.5
/usr/lib/python2.5/lib-dynload
/usr/lib/python2.5/lib-dynload/gdbm.so
/usr/lib/python2.6
/usr/lib/python2.6/lib-dynload
/usr/lib/python2.6/lib-dynload/gdbm.so
...

So it looks like this single package installs 3 .so libraries, one for each version of python.

Python2.6 is the default version of python in Ubuntu 9.10, but it is also possible to install python2.4, 2.5 and/or 3.0. Unless you do so, only /usr/lib/python2.6/lib-dynload/gdbm.so is used, the others are just wasting space.

Since the unneeded files in python2.4, 2.5, 3.0 are not very large, the package maintainer probably felt it was easier to ship one package rather than one for each version of python.

However, unless you know how to fix future apt-get errors, I'd recommend not manually deleting any files that were installed by packages in Ubuntu.

Answer 3

The short answer to your question: when you install packages from source, you should use the packages' setup.py installer to install them automatically and correctly. This installer already knows where to properly install the modules so Python can find them. To use, simply call with the exact Python interpreter you want to use the package with.

A crash course in setup.py. First, run it with the exact Python executable that you want the package to be available to. If you want to use the package with /usr/bin/python2.5, you should use /usr/bin/python2.5 to run setup.py. Second, go to the directory where that package's setup.py is installed. Third, you must install as root, so it's easiest to do the whole tihng as root. Fourth, if you want to install to multiple Python interpreters, you should run setup.py with each, but you should clean it in between. So here's what I would do:

% cd /root/directory/of/untarred/source/package
% sudo su
# /path/to/first/python setup.py build install
# rm -rf build
# /path/to/second/python setup.py build install
# rm -rf build
# exit
%

If you're installing modules by hand... you shouldn't, you should use its setup.py. (If you wrote a new module, you should write a setup.py for it.) If you must install by hand, you'll need to figure out which is the proper directory to install into for each Python, either by exploration and experimentation, or by calling into the same libraries that the installer calls to determine the proper directory. Installers using distutils call distutils.sysconfig.get_python_lib(); installers using setup_tools look in setup_tools.command.easy_install.easy_install.INSTALL_SCHEMES[os.name]["install_dir"].

Regarding dist-packages: I had a conversation with the maintainer of the Python package for Debian earlier this year. He'd implemented this dist-packages in the beta packages picked up by Ubuntu 9.04, but the code had a bug wrt PYTHONUSERBASE which I tripped over. We wound up talking a little. IIRC the reason for dist-packages had something to do with forcing the user to install packages in a different directory from apt-get. I clearly don't really understand the motivation, though, because in practice both the user and apt-get still install into the same directory.

lib-dynload isn't a Debian thing; that's a directory Python itself installs. I believe it was a directory just for shared libraries implementing modules. I'm not sure Python still uses it.

Finally, I don't know what you mean by "only python2.6 is actually working". What about these differently-named directories is "not working"?

Answer 4

I'm not sure what you mean by "Only python 2.6 is actually working." Suppose you run the "terminal emulator" and get a command-line prompt. Is this what you mean:

% python -V
Python 2.6

In other words, when you run Python, you get version 2.6? Well, have you tried this:

% python2.4

If Python 2.4 is correctly installed on your system, it will run. Likewise python2.5 will run Python 2.5.

If these don't run, and that is what you meant by "Only python 2.6 is actually working.", then one thing to try is to make sure that you actually have Ubuntu packages installed for Python 2.4 and Python 2.5.

% sudo apt-get install python2.4 python2.5

If you didn't have them installed before, this should add them. My thought is that you might have various libraries to support the older versions of Python, but you just don't have the actual Ubuntu packages for those older versions.