What do you use for a complex build process?

Question

I am trying to revamp our build process, which is currently a gigantic Ant build.xml that calls into other ant build files and executes several Java classes to perform more complex logic that would be impossible/scary to attemp in Ant.

Background:

• experience in Java and Ant, some Groovy
• Windows platforms

Goals:

• run as a combination of command line cron and when a servlet is posted to
• as simplified as possible, fewest languages and bouncing between techs

I need higher level logical power that a language like Java provides and Ant is pretty easy and we use the filtering to override default properties files for different clients. Mostly I'm wondering if there is something other than Ant/Java that people use.

Answer 1

I like using Rake as you can fall back on the power of the whole Ruby language and it's framework library when needed.

Answer 2

I use Ant, taking advantage of its macro feature. If you layout your project in a consistent mannner, you can eliminate a lot of the duplication by writing macros.

I've been building up an Antlib containing macros and custom tasks that I reuse across multiple projects.

Alternatively, some people swear by Maven. Other people just swear about Maven.

Answer 3

Except the Ant you mentioned and the scarry make/autotools, the mainstream tools are:

• SCons
• Jam
• CMake
• Maven

I use SCons, because it is python based, well-funded and elegant. Jam seems to be the most pragmatic one. I don't know too much about CMake. Maven may be the choice for you as it is Java centric and more high level than Ant.

More you can find at wikipedia: List of built tools

Answer 4

Also take a look at

• Gant
• Gradle

While more general purpose build systems like SCons are very powerful, the Java support is somewhat limited in comparison to systems specially tailored to build Java projects.

Answer 5

I use Maven, not just for build, I also use their release/dist plugin.

In a pair of commands I can have code that was in a source control to build, package and release.

The release plugin handles updating the version numbers, dist handles how to put everything together and zip it.

Ant looks hard when compared to Maven. Sure there is a learning curve with Maven, but reading a pom.xml is far easier than reading a build.xml.

Maven needs to be far less verbose.

Answer 6

I like Ant but only if you spend the time to write your own Java plugins to encapsulate complex actions. It isn't hard but unfortunately most people try to write their logic in XML w/the ant-contrib stuff. I think this is a huge mistake.

I've heard good things about rake and the groovy tools (mentioned in another comment) but I've got no experience with them.

If you're trying to script together several steps in a life-cycle you might be better off using a process automation based build server like AnthillPro (Cruise, BuildForge and Electric-Commander are the others in this space).

Another place to ask this kind of question is the CITCON mailing list. CITCON is a conference on Continuous Integration and Testing and the associated mailing list has turned into a really great community around these kinds of topics.

(I'm an organizer for CITCON but a labor of love not a profit maker. It really does have a really helpful mailing list. If I was pimping something for money it would be The CI Guys. ;-) )

Answer 7

If you pursue Maven, then you will have two problems: a complex build and learning the f@*#ing "magic" of Maven. Maven just makes the problem worse because it is obtuse and overly-complicated.

I inherited a legacy Maven 1.x build at a large Fortune 500 company. I used Maven 2.x by choice on many other projects in recent years. I evaluated Maestro, in hopes that it might make Maven tractable. My conclusion, like many other peoples' (check the 'net), is that Maven is a big step in the wrong direction. It definitely is not an improvement over Ant.

I have used Ant for MANY years, including writing a large open-source library of Ant helper scripts. I have also extensively used its .NET cousin nAnt. However, Ant has two major failings. One, XML is simply not the right place to be doing build tasks. Two, Ant and XML do not scale well to large, complex builds. In fact, I have written a lot here at SO about my experiences in that arena (and with Maven).

Industry leaders have concluded that a build is just another application, and should be approached using general application tools. However, since it involves system-level and cross-platform functionality, most development languages/platforms are not properly suited (which includes Java, and therefore Ant and Maven). That also excludes .NET.

I spent two years looking for an alternative, and I found it: Python. It has the right combination of system-level access, cross-platform portability, simplicity, readability, power, robustness, and maturity. SCons, buildbot, setuptools/easyinstall, and base Python are my current target platform for the build process. When necessary, integration with Ant, Maven, and any other such tool is easy. Meanwhile, I can use these tools for the core of any build on any platform with any source language. No more roadblocks, no more crazy complexity, no more supposedly-helpful "declarative" scripting, no more black-box f@*#ing "magic".

If you can't switch to Python, then try Ant + Ivy (at apache.org). It gives you Maven's cool repository without most of Maven's evils. That is what I am doing as well, where necessary and suitable.

Best wishes.

Answer 8

We use luntbuild. It's a web app that's very easy to use. It will check out from CVS/SVN, automatically increment the version/build number, execute build tasks in ant scripts and tag your repository with the new version. Can schedule automatic builds, have it email you or contact you via IM, also has security and build dependencies.

We're still using version 1.2.3 but I see it's up to 1.6.0. It just works.

http://luntbuild.javaforge.com/

EDIT: Re-reading your question I see now that you're looking for something to replace Ant. In our case Ant is not really a problem. We have projects setup in Netbeans, which uses Ant for building and we just have to implement some hooks into the existing scripts provided by Netbeans which is very easy to do.

EDIT: Looks like you can call Ant from Groovy. That would be nice because then you can re-use all the tasks that already exist for Ant.

http://groovy.codehaus.org/Using+Ant+from+Groovy

Answer 9

I use Rake everywhere I can. Where you need to build java code you might use it with jruby, or look at something like: buildr

Answer 10

Stick with Ant since you're building Java. I've mixed Ant/SCons for some JNI work, but in general I'd stay with Ant especially since you've got an existing build setup in Ant. Porting to Maven will be like shoving a square peg through a wall with no holes.

Embrace your custom Java logic for any and consider writing proper Ant tasks instead of executing external Java code. I solved some very complex parts of our build process by simply extending Ant to do exactly what I needed, ex. managing icon resources for a large gui project or injected subversion information directly info jar manifests (thank you SVNKit)

Answer 11

Try FinalBuilder

It provides a GUI interface to whatever your build process may be and has a native action for almost everything, plus an action studio where you can create your own.

With a bit of planning can be very streamlined.

Answer 12

I'd go with Ant any day of the week.

It's not perfect; XML is very verbose and implementing any logic at all is almost impossible, but even the most junior engineer on the team can at least understand what the ant file is doing within a day.

Complex logic can be refactored using java and integrated in ant, if you so wish. Ant gives you all the power of java:)

Dependency resolution is difficult no matter what system you use. With ant, the best solutions seem to be either a lib directory in which all your jars are stored, or an internal web server from which the libraries are copied at build time.

I also have some experience with both Maven 1 and Maven 2. That experience left me with the feeling that Maven is awesome for hobby projects, but can have complications for software you need to maintain over time.

I see two important problems with Maven:

• Any dependency you import using Maven may change over time without you knowing it, resulting in weird problems.
• You import the licenses of not only the software you import directly using Maven, but also the licenses used by the libraries which are indirectly imported

In short, I dislike the fact that the build depends on the time it is started. That can be a true problem when producing a bugfix release a year after the production release.

These problems can of course be managed (maybe using a nexus proxy) but you need to consider them before rebuilding the build system. At my company we decided to use Ant for all new project and try to port maven 1 and 2 to ant whenever the occasion presents itself. It's just too difficult to keep it working.

My advice, if you and your team know how to deal with ant, try to refactor your ant file and don't jump on some other build tool. It just takes too much time to get it right; time you could spend making money and surviving as a company :)

Answer 13

I have to add to these one solution that I find I can't live without ... its called Hudson. It only takes a few seconds to setup and you can usually get away with most of whatever your existing ANT files are already doing.

Additionally, Hudson provides a great way to "cron" builds, execute test cases, and generate "artifacts" (e.g. build products) in a way that anyone can download.

It is hard to capture all that it can do for you ... so just try it out ... you will no be disappointed.

Answer 14

Maven is really good choice to do large builds...in the majority of the cases where it doesn't work is very simple. The people misunderstand the concepts of Maven. If your are working with the "maven way" you will end up with smaller modules which gives you a better architecture in your software. On the other hand things like Hudson will support you in reducing build times by using Maven, cause Hudson supports to build only changed modules which is not supported by any other build tool. The problem with Maven is to learn and understand the concepts of Maven for example the structure of a project(folders etc.) or only a single artifact etc. The build-cycle will support you in different areas: compiling, packaging, deployment and release which is not supported by other tools (only if you implement it by hand...I've wrote many large Ant scripts to reach this)...Other problem like changes over the time are caused by ignoring the best practice and pin pointing versions which are used.