Which is the most pythonic: installing python modules via a package manager ( macports, apt) or via pip/easy_install/setuptools

views:

189

answers:

+3 Q:

Which is the most pythonic: installing python modules via a package manager ( macports, apt) or via pip/easy_install/setuptools

Usually I tend to install things via the package manager, for unixy stuff. However, when I programmed a lot of perl, I would use CPAN, newer versions and all that.

In general, I used to install system stuff via package manager, and language stuff via it's own package manager ( gem/easy_install|pip/cpan)

Now using python primarily, I am wondering what best practice is?

+7 A:

The system python version and its libraries are often used by software in the distribution. As long as the software you are using are happy with the same versions of python and all the libraries as your distribution is, than using the distribution packages will work just fine.

However, quite often you need development version of packages, or newer version, or older versions. And then it doesn't work any more.

It is therefore usually recommeded to install your own Python version that you use for development, and create development environments with buildout or virtualenv or both, to isolate the system python and the development environment from each other.

Lennart Regebro 2009-10-13 10:41:50

+8 A:

There are two completely opposing camps: one in favor of system-provided packages, and one in favor of separate installation. I'm personally in the "system packages" camp. I'll provide arguments from each side below.

Pro system packages: system packager already cares about dependency, and compliance with overall system policies (such as file layout). System packages provide security updates while still caring about not breaking compatibility - so they sometimes backport security fixes that the upstream authors did not backport. System packages are "safe" wrt. system upgrades: after a system upgrade, you probably also have a new Python version, but all your Python modules are still there if they come from a system packager. That's all personal experience with Debian.

Con system packages: not all software may be provided as a system package, or not in the latest version; installing stuff yourself into the system may break system packages. Upgrades may break your application.

Pro separate installation: Some people (in particular web application developers) argue that you absolutely need a repeatable setup, with just the packages you want, and completely decoupled from system Python. This goes beyond self-installed vs. system packages, since even for self-installed, you might still modify the system python; with the separate installation, you won't. As Lennart discusses, there are now dedicated tool chains to support this setup. People argue that only this approach can guarantee repeatable results.

Con separate installation: you need to deal with bug fixes yourself, and you need to make sure all your users use the separate installation. In the case of web applications, the latter is typically easy to achieve.

Martin v. Löwis 2009-10-13 10:57:17

Fair enough. I always wonder why setuptools and especially rubygems et al does not install into /usr/local tree, which is the place for things that are , well, local. Ah well. For what it is worth, being a sysadmin as well ( are we not all many things? ) I prefer your approach. Seems to be pretty problematic with python though. Holding my nose and going the "development environment route"

chiggsy 2009-10-13 11:19:23

I completely agree that distro packages are preferable when installing in the system Python. But that for development work, and also a production system that does a lot, you'll want separate environments for separate softwares (unless all of them come as updated and maintained distro packages).

Lennart Regebro 2009-10-13 12:24:48

As a personal policy, I abstain from using software for which there is no system package, or from a version different from the one the system packager provides. This may mean that I cannot use a lot of packages - tough luck; I'll may have to reimplement stuff. On the plus side, I'm restricted to using "stable" software, so I actually don't need to track each and every development release just because some library decides that it requires an unreleased version of some other package.

Martin v. Löwis 2009-10-13 12:58:35

I fall into the pro separate installation category, but I wouldn't say you "absolutely" need a repeatable setup. It's just that once you have one, a lot of sysadmin and debugging headaches go away. Knowing that every library being used in production is *exactly* the same version as what's in my dev sandbox makes things easier. Similarly, knowing that upgrading a library on one app in production won't affect any of the other production apps (adversely or otherwise) is comforting to the sysadmin in me.

thraxil 2009-10-13 14:56:40

This is a pretty fair summary of a debate that's been hashed over too many times. It comes down to what your needs are. Web app developers (and I'm one) have very good reasons to want repeatable isolated deploys, and to need recent versions of things that may not be OS-packaged. I'm very happy with virtualenv+pip to do that for me in my web-dev work. At the same time, I'm very happy to use only stable OS-packaged stuff for desktop apps etc.

Carl Meyer 2009-10-26 16:22:22

ansaurus

tags:

views:

answers:

Which is the most pythonic: installing python modules via a package manager ( macports, apt) or via pip/easy_install/setuptools

related questions