views:

181

answers:

3

Our production machines are running on debian etch. Now, they finally released lenny, the day will come we need to upgrade these systems. How can I do this with minimal risk? Are there any premises, preparations of fall-back scenarios and do I need a plan B in case something goes wrong? Besides the binary packages handled by the debian installer there are a couple of compiled applications running on the machines.

+1  A: 

Most people don't update production servers for exactly this reason - if it's working correctly, you wouldn't update unless you had a compelling reason.

Assuming you have a dev box built similarly to the production machine, you can simulate the update on the dev box.

Steve B.
+2  A: 

Having just gone through that transition for some dev boxes, I wanted to point out that you'll probably want to recompile any custom libraries that you'll be linking against. Lenny uses GCC 4.3, whereas Etch uses 4.1. The output from either compiler isn't very compatible with the other. You may need to install the gcc-4.1 package to do things like compile custom kernel modules.

If you're using 3rd party tools that have a plugin interface, you may have challenges there. I've been having troubles getting Matlab plugins (mex files) to work.

I'd suggest starting with a test system. After hammering it for a while and verifying that everything's working, switch it to be a production box.

Mr Fooz
+6  A: 

Personally I wouldn't upgrade any OS on an important server. OS upgrades always have the potential for subtle bugs, whether it's Windows, Linux or anything else. Debian has got better than it used to be in this regard; dist-upgrade doesn't hose the machine nearly as often as it used to back in the day. But for production machines there is no point in risking it.

Set up new servers with a fresh OS and application deployment and swap them in as needs arise. There is no need to hurry to replace Etch companywide in one go. It will be supported with security updates for a while yet.

bobince
In principle, this is a nice idea, but not everyone has the budget for it. Even if the extra servers are cheap, the manpower (including that needed to make the purchase if you're in a larger organization with onerous purchasing procedures) to do this can be a problem in a small shop.
PTBNL