How do you handle multiple jar versions with lots of FOSS/COTS?

Question

We have a Java product that uses a lot of FOSS/COTS software. A number of our "externals" use the same jar product, but a different version. For example, Ant 1.6.5 and Ant 1.7.0; or multiple versions of xerces. What's I'm concerned about is that the behavior of our application may change or worse, epic fail if we change the order in which the classpath is put together. We use vbs scripts to set environment variables for each product's classpath, and then Ant xml files which reference those environment variables.

So, a couple of questions:

1. How do I manage multiple versions of the same jar when using so many different externals? Surely I can't just find all of the unique jars and put them on one big classpath - or can I?
2. Is there a smarter way to put our build dependencies (and classpath) together?

One positive step is that I'm planning on using wildcards to grab all of the jars. But it's really just the import order problem that I'm mostly concerned about.

Note: Don't shoot the messenger. This system was put into place several years ago, long before I got here. I'm just the cleanup man.

Answer 1

Maven. Takes a while to figure it out and get it set up, but it is pretty magical once it is working.

See http://maven.apache.org/download.html

Answer 2

You could add Ivy into your ant build to explicitly version dependencies. When it comes to building a classpath, I'd suggest you strive for a situation where you don't have multiple versions of the same library. We had some very annoying behaviour where the IDE would build the classpath using transitive dependencies, but we build our unix classpath alphabetically which resulted in one of Saxon/Xalan JavaMail/Genronimo-JavaMail being loaded depending on where you ran the code.

As mentioned in the other answer, if you have time to re-work your build system you should look at maven.

I quite like this talk by John Smart on maintaining your build environment

Answer 3

If you have to put all the jars in the same class loader, then you can use maven to determine the set of dependencies. This will select the latest version of each library required by the set of dependencies. (E.g. Ant 1.7.0 will be selected, and not 1.6.5.) This scheme works well - unless a library is not backwards-compatible with an earlier version. Consequently, it's a good idea to test the relevant functions/features of your app when changing dependencies.

An alternative, which is only practical if the libraries can be split into interface and implementation, is to load the interfaces in a common class loader, and the implementations+dependencies in a custom class loader. This isolates each dependency by giving it it's own classloader. This is essentially what OSGi does.

Answer 4

This is what OSGi was designed to solve: Having multiple independent versions of the same jars in your application.

Answer 5

I'm not sure that the suggestions to use Maven or Ivy will solve the real problem. They may clean up declaration of dependencies but they won't do anything about conflicts. If you need to ship two different versions of the same library (because using the latest version will not necessarily work if the library is not backwards-compatible), you can use jarjar to repackage libraries with different package names so that both versions can be loaded if necessary.