What are good Linux distributions (and methods) for developing the latest Linux upstream (kernel, Gnome etc.) packages?

Question

The people who really know the answer to this question are those who develop for programs or libraries which are included in Linux distributions, and which depend on other libraries which are included in Linux distributions. Them just telling me what they do in terms of the development environment I'm describing would be helpful.

The problem basically is thus:

Linux distributions bundle the Linux kernel with typical Linux plumbing - glib, dbus, udev and the like. Most also bundle desktop environments like KDE or Gnome.

While the distribution may have a few bundling program packages of its own, most of programs and libraries come from upstream, meaning sources other than itself. Packages like dbus are "upstream" and flow downstream to distributions such as Debian, Fedora, Ubuntu and the like.

Sometimes I use these programs and feel they can be improved. Some of the improvements are bug fixes that can't really be argued with - say a small patch that prevents a segmentation fault from an unneeded memory violation.

When building these packages, one thing distributions include are "depends" lists, as many of these programs rely on certain libraries (and certain versions of those libraries), and some of those libraries rely on other libraries. This leads to some of the complications I am encountering.

If I encounter a bug in a program or library and report it upstream, a common question is, "are you using the latest version in source control, or an older version". Understandably, the developers want me using the code from the latest commit, as they don't want to waste time retracing ground they have already covered, especially if the bug has been fixed.

So I go to get the latest commit. But then I encounter a problem because the latest commit now depends on a library which is newer than my operating system uses.

An example might be is if I am looking at a PDF with the default reader in Ubuntu, evince. It is not displaying properly due to what appears to be a flaw in the poppler library evince depends on. So I post a bug, but they want to know if I have the latest poppler version. So I download the latest poppler commit, but the poppler library depends on the cairo library, and my OS does not have the latest cairo library. So I have to get that - but then cairo complains I don't have the latest pixman and x11 libraries.

This is one example of this, but it goes for pretty much any package.

So my question is, what is a good Linux distribution to use for this sort of thing? One which is known for having the most up-to-date packages, or at least the option to get them easily?

Or maybe a Linux distribution that more easily allows me to do such things. I have heard talk about how Gentoo builds from source via emerge, and I could see this solving a lot of problems.

There's also the option of having a subsystem of the latest packages in a chrooted environment within another system using tools such as pbuilder and debootstrap.

Lastly, I could just download the latest commits of all libraries, build the one with no dependencies I'm concerned with first, and then go to the next one and build that, pointing the program to use my compiled local library, not the system library, and then do that again and so on. This is what I'm trying to avoid for a number of reasons (way too much manual mucking around, I might miss an important dependency etc.)

Answer 1

Gentoo is pretty good about bleeding edge packages, as long as you unmask "~x86" (or ~amd64 or ~arch for whatever your arch is).

ebuilds can integrate with version control, so it's not too difficult to make a modified ebuild pull the very latest revision form svn or git and then compile and install it just as if it was the distributed package.

Answer 2

Get involved with a distro, focus on the packages you want to improve
Distros ensure that package versions work well together. This is the first job in building a Linux system and they have already done it. They ensure that everything "fits". If you want to improve a package, then do the improvements in the distro's version. If the patch holds value, it may get applied to the latest version.

Not all packages suffer from depenancy issues, so you can build those from the repo and contribute that way. Primarily, the distro is the environment, with upstream packages often getting contributions from distro developers.

There isn't much point building an entire distribution just to develop for the Gnome core really. Just improve say, Gentoo's Gnome build and try to get applicable patches upstream.

Focus on where you can input value, and not spend 1/4 of your life waiting for builds and working out why package X won't compile against library Y when you only want to work on Z anyway!

I work with Fedora Rawhide when doing any dev-stuff, it isn't as bleeding edge as some, but offers good stability and changeability IMVHO

Or there is always:

Linux From Scratch, if you like pain.

Answer 3

It shouldn't really matter. A distro should give you a basic backbone and some stability -- so use whichever distro "works" for you. And whatever you work on may then come from a revision control system anyway.

That said, I started with Debian in 1994 and still use it as well as Ubuntu (and I've been maintaining packages for Debian since 1995 too). When I develop on Ubuntu I sometimes take a source package from Debian unstable and build it locally -- which is actually pretty simple yet really useful. That way I have the best of both worlds: a "reasonably current" set of everything via bi-annual Ubuntu upgrades, plus selective insertions of whatever I need that is bleeding edge. A very good alternative is Debian testing which I have on other systems.

Another choice is virtualization: just add a kvm, or xen, or virtualbox instance of a headless 'bleeding edge' distro (Debian unstable, say) for testing is useful too.

Answer 4

Gentoo is useful if you need or want to edit the source code of a package. However, if you only need the binaries, Arch Linux is a good choice, and it's well documented. If you want more control, you can always download and compile by yourself the packages, but I see you only need up-to-date packages so Arch would be better. It's up to you.

Answer 5

The choice of distro shouldn't matter much. Either Debian unstable, Fedora rawide or Mandriva Cooker versions should be good for a development environment of Linux packages.