views:

1015

answers:

7

We have a large codebase that takes approx 12 minutes on the developer machines to auto-generate some Java 5 classes using JavaCC and then compiles all the classes as well as running the units test.

The project consists of multiple projects which can be built in groups, but we are aiming for a full a build in under 10 minutes

What tips are there for reducing this build time?

Thanks

+1  A: 

One quick fix that might shave some time off is to ensure that you are running Ant using the server JVM (by default it uses the client VM). Set ANT_OPTS to include "-server".

Dan Dyer
Will this speed up times on a desktop machine as well as the server?
Craig Angus
Yes, this simply changes the way the JIT optimizes bytecode when running Ant.
Alexander
On Windows I knocked about a minute off a 6 minute build with this switch.
Dan Dyer
It might not make a significant difference, but it's worth trying since it's trivial to do.
Dan Dyer
+2  A: 
  • Profile the build process and see where the bottlenecks are. This can give you some ideas as to how to improve the process.
  • Try building independent projects in parallel on multi-core/CPU machines. As an extension of this idea, you may want to look around for a Java equivalent of distcc (don't know whether it exists) to distribute your build over a number of machines.
  • Get better machines.
Alexander
"Get better machines." we have recently all converted to Linux as the build times went from approx 40 minutes to approx 12 minutes.
Craig Angus
"Try building independent projects in parallel on multi-core/CPU machines. " Most of are on Dual core machines and we have a parallel build process that is slightly faster than the normal build
Craig Angus
A: 

This probably wouldn't help in the very near term, but figured I should throw it out there anyway.

If your project is breakable into smaller projects (a database subsystem, logging, as examples), you may be interested in using something like maven to handle the build. You can run each smaller bite as a separate project or module, and maven will be able to maintain what needs to be built if changes exist. In this the build can focus onthe main portion of your project and it won't take nearly as long.

Spencer K
A: 

What is the breakdown in time spent:

  1. generating the classes
  2. compiling the classes
  3. running the tests

Depending on your project, you may see significant increases in build time by allocating a larger heap size to javac(memoryMaximumSize) and junit(maxmemory).

npellow
+1  A: 

Try be inspired by pragmatic programmer. Compile only what is necessary, have two or more test suites. One for quick tests, other for full tests. Consider if there is real need to use each build-step every time. It necessary try to use jikes compiler instead of javac. After project spans several hundreds of classes I switch to jikes to improve speed. But be aware of potential incompatibility issues. Don't forget to include one all in one target to perform every step with full rebuild and full test of project.

Rastislav Komara
+1  A: 

Now that you've explained the process in more detail, here are two more options:

  1. A dedicated machine/cluster where the build is performed much quicker than on a normal workstation. The developers would then, before a commit, run a script that builds their code on the dedicated machine/cluster.

  2. Change the partitioning into sub-projects so that it's harder to break one project by modifying another. This should then make it less important to do a full build before every commit. Only commits that are touching sensitive sub-projects, or those spanning multiple projects would then need to be "checked" by means of a full build.

Alexander
A: 

Is it very important that the entire build lasts less than 10 minutes? If you make the sub-projects independent from one another, you could work on one sub-project while having already compiled the other ones (think Maven or Ivy to manage the dependencies).

Another solution (and if your modules are reasonably stable) is to treat your sub-projects as standalone projects. Each project would then follow their own release cycle and be available from a local Maven/Ivy repository. This of course works well if at least parts of the project are reasonably stable.

Vladimir