views:

1088

answers:

8

I have a Java Maven project with about 800 source files (some generated by javacc/JTB) which is taking a good 25 minutes to compile with javac.

When I changed my pom.xml over to use the Eclipse compiler, it takes about 30 seconds to compile.

Any suggestions as to why javac (1.5) is running so slowly? (I don't want to switch over to the Eclipse compiler permanently, as the plugin for Maven seems more than a little buggy.)

I have a test case which easily reproduces the problem. The following code generates a number of source files in the default package. If you try to compile ImplementingClass.java with javac, it will seem to pause for an inordinately long time.

import java.io.File;
import java.io.FileNotFoundException;
import java.io.PrintStream;

public class CodeGenerator
{
    private final static String PATH = System.getProperty("java.io.tmpdir");
    private final static int NUM_TYPES = 1000;

    public static void main(String[] args) throws FileNotFoundException
    {
     PrintStream interfacePs = new PrintStream(PATH + File.separator + "Interface.java");
     PrintStream abstractClassPs = new PrintStream(PATH + File.separator + "AbstractClass.java");
     PrintStream implementingClassPs = new PrintStream(PATH + File.separator + "ImplementingClass.java");
     interfacePs.println("public interface Interface<T> {");
     abstractClassPs.println("public abstract class AbstractClass<T> implements Interface<T> {");
     implementingClassPs.println("public class ImplementingClass extends AbstractClass<Object> {");

     for (int i=0; i<NUM_TYPES; i++)
     {
      String nodeName = "Node" + i;
      PrintStream nodePs = new PrintStream(PATH + File.separator + nodeName + ".java");
      nodePs.printf("public class %s { }\n", nodeName);
      nodePs.close();
      interfacePs.printf("void visit(%s node, T obj);%n", nodeName);
      abstractClassPs.printf("public void visit(%s node, T obj) { System.out.println(obj.toString()); }%n", nodeName);
     }
     interfacePs.println("}");
     abstractClassPs.println("}");
     implementingClassPs.println("}");
     interfacePs.close();
     abstractClassPs.close();
     implementingClassPs.close();
    }
}
A: 

Perhaps the Eclipse build is only compiling modified source. What happens if you compile it in eclipse after a clean?

Jack BeNimble
I did a 'mvn clean' before 'mvn install' in both cases.
Simon Nickerson
A: 

For the Sun compiler you are starting up a whole JVM process for each file you wish to compile. For the Eclipse compiler it is just connecting to a daemon process. I suggest setting fork to false, although it still may not be quite as fast.

Tom Hawtin - tackline
Would that make a 25 minute difference? I wouldn't think so...
Michael Myers
25 minutes/80 sources files = 18.75 seconds per javac. Possible if for each source file it had to load a shedload of jars/classes.
Tom Hawtin - tackline
Thank you for your suggestion - I tried it, but unfortunately it did not make a noticeable difference in the compilation time.
Simon Nickerson
+2  A: 

The fact that you're using generated source, the massive difference in speed and the StackOverflowError might suggest that one (or more) of your files have some constructs that the javac parsers doesn't agree with.

Could you try to compile only subsets of your code and see if any one class/package slows down the process especially (probably one of the generated ones).

Joachim Sauer
I've now tried splitting the project in two - 391 files generated by javacc/jtb and 373 hand-coded. The vast majority of the time is spent in compiling the hand-coded ones (compiling the generated ones took about 18 seconds)
Simon Nickerson
Interesting, I wouldn't have guessed that. I'd split the source files further and further to try to find out if a few files are responsible to the slow down.
Joachim Sauer
Another hint: run your build using "strace -eopen ant". This will print out every time a file is opened. Wait for pauses in that huge stream of output and check the last filename opened before. Should give you a decent hint.
Joachim Sauer
+5  A: 

It may be that the javac compiler operates close at its heap limit (64MB or so). In that case, it spends most of the time in the garbage collector. Give the compiler a good chunk of memory, say 256M or 512M and see if it runs faster.

Ingo
Unless otherwise configured, the maven-compiler-plugin runs javac in-process. A possible fix for simonn to try would be to set the MAVEN_OPTS environment variable to "-Xms128M Xmx512M" or so.If the plugin is configured fork=true, he can use the meminitial and maxmem params to control this.
Barend
I have tried running "javac -verbose -J-Xms512m -J-Xmx1024m ImplementingClass.java", and the problem persists.
Simon Nickerson
A: 

I don't know how maven calls the compiler but the performance numbers you mention suggest that javac is executed in it's own process/VM as already suggested in another answer. As starting a new process/VM for every file you compile is very costly you need to ensure to configure the compiler to use the VM you might alreay have. I know ANT offers that, but I have not used maven myself. Given the fact that it is popular I doubt that it lacks such an important feature though.

lothar
A: 

I think something like the following is going on: Maven forks javac, JVM processes for separate steps in its life-cycle: Maven Build Life-cycle

Eclipse typically runs its compile in the background (on Save), so that step will be added to the compile phase. If there are substantial dependencies, this is where you're losing throughput.

In addition (depending upon mvn configuration) each test method gets its own JVM. Since test passage is a pre-req to the package phase, it's possible that you're losing time to your JUnit test executions (particularly if they're slow running tests). This is only a likely culprit if you have a lot of test code in your source tree.

Most likely of all, your class does substantial amounts of File I/O, so that's an area of opportunity. It looks like your loop is executing 1000 times per File discovery event meaning 800*1000 =800,000 PrintStream creations in the body of the loop.

jasonnerothin
+4  A: 

You get the same behaviour with JDK 1.6, including update 14, build 04, using G1 doesn't change the behaviour, (though G1 appears to work really well).

Monitoring javac with jvisualvm, repeated thread dumps show the main thread spending lots of time in

at com.sun.tools.javac.code.Types.isSubSignature(Types.java:1846)
at com.sun.tools.javac.code.Symbol$MethodSymbol.overrides(Symbol.java:1108)
at com.sun.tools.javac.code.Symbol$MethodSymbol.implementation(Symbol.java:1159)
at com.sun.tools.javac.comp.Check.checkCompatibleConcretes(Check.java:1239)
at com.sun.tools.javac.comp.Check.checkCompatibleSupertypes(Check.java:1567)
at com.sun.tools.javac.comp.Attr.attribClassBody(Attr.java:2674)
at com.sun.tools.javac.comp.Attr.attribClass(Attr.java:2628)
at com.sun.tools.javac.comp.Attr.attribClass(Attr.java:2564)
at com.sun.tools.javac.main.JavaCompiler.attribute(JavaCompiler.java:1036)
at com.sun.tools.javac.main.JavaCompiler.compile2(JavaCompiler.java:765)
at com.sun.tools.javac.main.JavaCompiler.compile(JavaCompiler.java:730)
at com.sun.tools.javac.main.Main.compile(Main.java:353)
at com.sun.tools.javac.main.Main.compile(Main.java:279)
at com.sun.tools.javac.main.Main.compile(Main.java:270)
at com.sun.tools.javac.Main.compile(Main.java:69)
at com.sun.tools.javac.Main.main(Main.java:54)

and churning through a large number of short lived instances of these classes:

com.sun.tools.javac.code.Types$Subst
com.sun.tools.javac.util.List
com.sun.tools.javac.code.Types$MethodType

I suspect the code is churning through com.sun.tools.javac.comp.Check.checkCompatibleConcretes comparing each method with every other method

That method's javadoc:

/** Check that a class does not inherit two concrete methods
 *  with the same signature.
 */

It may be that eclipse's compiler either doesn't perform that check, or doesn't perform it in the same way.

Stephen Denne
+7  A: 

Sun has confirmed to me by email that this is a new bug (6827648 in their bug database).

Simon Nickerson