How do you decide when to upgrade a library in your project ?

Question

I work on a project that uses multiple open source Java libraries. When upgrades to those libraries come out, we tend to follow a conservative strategy:

1. if it ain't broke, don't fix it
2. if it doesn't have new features we want, ignore it

We follow this strategy because we usually don't have time to put in the new library and thoroughly test the overall application. (Like many software development teams we're always behind schedule on features we promised months ago.)

But, I sometimes wonder if this strategy is wise given that some performance improvements and a large number of bug fixes usually come with library upgrades. (i.e. "Who knows, maybe things will work better in a way we don't foresee...")

What criteria do you use when you make these types of decisions in your project?

Answer 1

I've learned enough lessons to do the following:

1. Check the library's change list. What did they fix? Do I care? If there isn't a change list, then the library isn't used in my project.
2. What are people posting about on the Library's forum? Are there a rash of posts starting shortly after release pointing out obvious problems?
3. Along the same vein as number 2, don't upgrade immediately. EVERYONE has a bad release. I don't intend to be the first to get bit with that little bug. (anymore that is). This doesn't mean wait 6 months either. Within the first month of release you should know the downsides.
4. When I decide to go ahead with an upgrade; test, test test. Here automated testing is extremely important.

EDIT: I wanted to add one more item which is at least as important, and maybe more so than the others.

• What breaking changes were introduced in this release? In other words, is the library going off in a different direction? If the library is deprecating or replacing functionality you will want to stay on top of that.

Answer 2

Important: Avoid Technical Debt.

"If it ain't broke, don't upgrade" is a crazy policy that leads to software so broken that no one can fix it.

Rash, untested changes are a bad idea, but not as bad as accumulating technical debt because it appears cheaper in the short run.

Get a "nightly build" process going so you can continuously test all changes -- yours as well as the packages on which you depend.

Until you have a continuous integration process, you can do quarterly major releases that include infrastructure upgrades.

Avoid Technical Debt.

Answer 3

Some important questions:

• How widely used is the library? (If it's widely used, bugs will be found and eliminated more quickly)
• How actively developed is it?
• Is the documentation very clear?
• Have there been major changes, minor ones, or just internal changes?
• Does the upgrade break backwards compatibility? (Will you have to change any of your code?)

Unless the upgrade looks bad according to the above criteria, it's better to go with it, and if you have any problems, revert to the old version.

Answer 4

One approach is to bring the open source libraries that you use under your own source code control. Then periodically merge the upstream changes into your next release branch, or sooner if they are security fixes, and run your automated tests.

In other words, use the same criteria to decide whether to use upstream changes as you do for release cycles on code you write in house. Consider the open source developers to be part of your virtual development team. This is really the case anyway, it's just a matter of whether you choose to recognise it as part of your development practices.

Answer 5

While you don't want to upgrade just because there's a new version, there's another consideration, which is availability of the old version. I've run into that problem trying to build open source projects.

Answer 6

I usually assume that ignoring a new version of a library (coz' it doesn't have any interesting features or improvements) is a mistake, because one day you'll find out that this version is necessary for the migration to the next version which you might want to upgrade to.

So my advice is to review carefully what has changed in the new version, and consider whether the changes requires a lot of testing, or little.

If a lot of testing are required, it is best to upgrade to the newer library at the next release (major version) of your software (like when moving from v8.0 to v8.5). When this happens, I guess there are other major modifications as well, so a lot of testing is done.

Answer 7

I prefer not to let the versions lag too far behind on dependant libraries. Up to a year is ok for most libraries unless security or performance issues are known. Libraries with known security issues are a must for refreshing.

I periodically download the latest version of each library and run my apps unit tests using them. If they pass, I use them in our development and integration environments for a while and push to QA when I'm satisfied they don't suck.

The above procedure assumes the API hasn't changed significantly. All bets are off if I need to refactor existing code just to use a newer library version. (e.g. Axis 1x vs. 2x) Then I would need to get management involved to make the decision to allocate resources. Such a change would typically be differed until a major revision of the legacy code is planned.