Why would Java classloading fail on Linux, but succeed on Windows?

Question

I've got a Java web application (using Spring), deployed with Jetty. If I try to run it on a Windows machine everything works as expected, but if I try to run the same code on my Linux machine, it fails like this:

[normal startup output]
11:16:39.657 INFO   [main] org.mortbay.jetty.servlet.ServletHandler$Context.log>(ServletHandler.java:1145) >16> Set web app root system property: 'webapp.root' = [/path/to/working/dir]
java.lang.reflect.InvocationTargetException
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:597)
        at org.mortbay.start.Main.invokeMain(Main.java:151)
        at org.mortbay.start.Main.start(Main.java:476)
        at org.mortbay.start.Main.main(Main.java:94)
Caused by: java.lang.ExceptionInInitializerError
        at org.springframework.web.util.Log4jWebConfigurer.initLogging(Log4jWebConfigurer.java:129)
        at org.springframework.web.util.Log4jConfigListener.contextInitialized(Log4jConfigListener.java:51)
        at org.mortbay.jetty.servlet.WebApplicationContext.doStart(WebApplicationContext.java:495)
        at org.mortbay.util.Container.start(Container.java:72)
        at org.mortbay.http.HttpServer.doStart(HttpServer.java:708)
        at org.mortbay.util.Container.start(Container.java:72)
        at org.mortbay.jetty.Server.main(Server.java:460)
        ... 7 more
Caused by: org.apache.commons.logging.LogConfigurationException: org.apache.commons.logging.LogConfigurationException: No suitable Log constructor [Ljava.lang.Class;@15311bd for org.apache.commons.logging.impl.Log4JLogger (Caused by java.lang.NoClassDefFoundError: org/apache/log4j/Category) (Caused by org.apache.commons.logging.LogConfigurationException: No suitable Log constructor [Ljava.lang.Class;@15311bd for org.apache.commons.logging.impl.Log4JLogger (Caused by java.lang.NoClassDefFoundError: org/apache/log4j/Category))
        at org.apache.commons.logging.impl.LogFactoryImpl.newInstance(LogFactoryImpl.java:543)
        at org.apache.commons.logging.impl.LogFactoryImpl.getInstance(LogFactoryImpl.java:235)
        at org.apache.commons.logging.impl.LogFactoryImpl.getInstance(LogFactoryImpl.java:209)
        at org.apache.commons.logging.LogFactory.getLog(LogFactory.java:351)
        at org.springframework.util.SystemPropertyUtils.(SystemPropertyUtils.java:42)
        ... 14 more
Caused by: org.apache.commons.logging.LogConfigurationException: No suitable Log constructor [Ljava.lang.Class;@15311bd for org.apache.commons.logging.impl.Log4JLogger (Caused by java.lang.NoClassDefFoundError: org/apache/log4j/Category)
        at org.apache.commons.logging.impl.LogFactoryImpl.getLogConstructor(LogFactoryImpl.java:413)
        at org.apache.commons.logging.impl.LogFactoryImpl.newInstance(LogFactoryImpl.java:529)
        ... 18 more
Caused by: java.lang.NoClassDefFoundError: org/apache/log4j/Category
        at java.lang.Class.getDeclaredConstructors0(Native Method)
        at java.lang.Class.privateGetDeclaredConstructors(Class.java:2389)
        at java.lang.Class.getConstructor0(Class.java:2699)
        at java.lang.Class.getConstructor(Class.java:1657)
        at org.apache.commons.logging.impl.LogFactoryImpl.getLogConstructor(LogFactoryImpl.java:410)
        ... 19 more
Caused by: java.lang.ClassNotFoundException: org.apache.log4j.Category
        at java.net.URLClassLoader$1.run(URLClassLoader.java:200)
        at java.security.AccessController.doPrivileged(Native Method)
        at java.net.URLClassLoader.findClass(URLClassLoader.java:188)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:252)
        at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:320)
        ... 24 more
[shutdown output]

I've run the app with java -verbose:class, and according to that output, org.apache.log4j.Category is loaded from the log4j JAR in my /WEB-INF/lib, just before the first exception is thrown.

Now, the Java versions on the two machines are slightly different. Both the machines have Sun's java, the Linux machine has 1.6.0_10, while the Windows machine has 1.6.0_08, or maybe 07 or 06, I can't remember the exact number right now, and don't have the machine at hand. But even though the minor versions of the Javas are slightly different, the code shouldn't break like this. Does anyone understand what's wrong here?

Answer 1

You're using the same WAR on both machines? Have you checked if the WAR files are identical (no transfer errors occured)?

Answer 2

Some random things to consider:

(1) Check if there are any other versions of log4j floating around on the linux instance, outside of the web-app directories?

(2) Is apache commons logging being used at all? You might want to consider SLF4J instead?

(3) Did the JAR/WAR become corrupt in some way - was it FTP'ed in ASCII or Binary?

(4) Print out the classloader hierarchy in each case, just to see if there are any discrepancies?

Answer 3

This seems like a classic classloader problem. It could be due to another web app being loaded first which also uses log4j but a version that is different to the one used by the app you are testing. The classloader uses the first version of the class it finds. The server class loading policy can usually be changed in the config files. Sorry I am a bit rusty on this but maybe it can point you in the correct direction.

1. Make sure there are no other installed apps on the web server,
2. Make sure the log4j being loaded is the correct version,
3. Make sure you don't have a log4j lurking somewhere in the classpath of the server.

HTH

Answer 4

You must understand that a classloader can't see everything; they can only see what a parent classloader has loaded or what they have loaded themselves. So if you have two classloaders, say one for Jetty and another for your webapp, your webapp can see log4j (since the JAR is the WEB-INF/lib) but Jetty's classloader can't.

If you manage to make a class available to Jetty (for example something in the DB layer) which uses log4j but which ends up running in the context (and classloader) of Jetty, you will get an error.

To debug this, set a breakpoint in org.springframework.web.util.Log4jWebConfigurer.initLogging(). If you can, copy the source of this class into your project (don't forget to delete it afterwards) and add this line:

ClassLoader cl = Thread.currentThread().getContextClassLoader();

Have a look at the cl object in your debugger. That should give you some information who created it. My guess is that this is the classloader from Jetty.

[EDIT] Note that you get in a different mess if you have log4j in both classloaders: In this case, you will have two classes with the same name which create objects which are not assignment compatible! So make sure there is only a single instance of this jar or that instances of log4j will never be passed between the two contexts (which is usually not possible).

Answer 5

Even though the original problem was solved for the asker, I'll point out that a common source of problems when running the same code on Windows vs Linux (or Unix) is case-sensitivity issues. Windows ignores case while Linux or Unix is case-sensitive. This has bitten me more than once.

So if you specify a jar or directory on the classpath, but it isn't the right case then it will fail on Linux but succeed on Windows. This can also be the source of FileNotFoundExceptions.