Tips for speeding up build time on Linux using ANT, Javacc, JUnit and compiling Java classes

Question

We have a large codebase that takes approx 12 minutes on the developer machines to auto-generate some Java 5 classes using JavaCC and then compiles all the classes as well as running the units test.

The project consists of multiple projects which can be built in groups, but we are aiming for a full a build in under 10 minutes

What tips are there for reducing this build time?

Thanks

Answer 1

One quick fix that might shave some time off is to ensure that you are running Ant using the server JVM (by default it uses the client VM). Set ANT_OPTS to include "-server".

Answer 2

• Profile the build process and see where the bottlenecks are. This can give you some ideas as to how to improve the process.
• Try building independent projects in parallel on multi-core/CPU machines. As an extension of this idea, you may want to look around for a Java equivalent of distcc (don't know whether it exists) to distribute your build over a number of machines.
• Get better machines.

Answer 3

This probably wouldn't help in the very near term, but figured I should throw it out there anyway.

If your project is breakable into smaller projects (a database subsystem, logging, as examples), you may be interested in using something like maven to handle the build. You can run each smaller bite as a separate project or module, and maven will be able to maintain what needs to be built if changes exist. In this the build can focus onthe main portion of your project and it won't take nearly as long.

Answer 4

What is the breakdown in time spent:

1. generating the classes
2. compiling the classes
3. running the tests

Depending on your project, you may see significant increases in build time by allocating a larger heap size to javac(memoryMaximumSize) and junit(maxmemory).

Answer 5

Try be inspired by pragmatic programmer. Compile only what is necessary, have two or more test suites. One for quick tests, other for full tests. Consider if there is real need to use each build-step every time. It necessary try to use jikes compiler instead of javac. After project spans several hundreds of classes I switch to jikes to improve speed. But be aware of potential incompatibility issues. Don't forget to include one all in one target to perform every step with full rebuild and full test of project.

Answer 6

Now that you've explained the process in more detail, here are two more options:

1. A dedicated machine/cluster where the build is performed much quicker than on a normal workstation. The developers would then, before a commit, run a script that builds their code on the dedicated machine/cluster.

2. Change the partitioning into sub-projects so that it's harder to break one project by modifying another. This should then make it less important to do a full build before every commit. Only commits that are touching sensitive sub-projects, or those spanning multiple projects would then need to be "checked" by means of a full build.

Answer 7

Is it very important that the entire build lasts less than 10 minutes? If you make the sub-projects independent from one another, you could work on one sub-project while having already compiled the other ones (think Maven or Ivy to manage the dependencies).

Another solution (and if your modules are reasonably stable) is to treat your sub-projects as standalone projects. Each project would then follow their own release cycle and be available from a local Maven/Ivy repository. This of course works well if at least parts of the project are reasonably stable.