What's the best Scala build system?

Question

I've seen questions about IDE's here -- Which is the best IDE for Scala development? and What is the current state of tooling for Scala?, but I've had mixed experiences with IDEs. Right now, I'm using the Eclipse IDE with the automatic workspace refresh option, and KDE 4's Kate as my text editor. Here are some of the problems I'd like to solve:

1. use my own editor IDEs are really geared at everyone using their components. I like Kate better, but the refresh system is very annoying (it doesn't use inotify, rather, maybe a 10s polling interval). The reason I don't use the built-in text editor is because broken auto-complete functionalities cause the IDE to hang for maybe 10s.
2. rebuild only modified files The Eclipse build system is broken. It doesn't know when to rebuild classes. I find myself almost half of the time going to project->clean. Worse, it seems even after it has finished building my project, a few minutes later it will pop up with some bizarre error (edit - these errors appear to be things that were previously solved with a project > clean, but then come back up...). Finally, setting "Preferences / Continue launch if project contains errors" to "prompt" seems to have no effect for Scala projects (i.e. it always launches even if there are errors).
3. build customization I can use the "nightly" release, but I'll want to modify and use my own Scala builds, not the compiler that's built into the IDE's plugin. It would also be nice to pass [e.g.] -Xprint:jvm to the compiler (to print out lowered code).
4. fast compiling Though Eclipse doesn't always build right, it does seem snappy -- even more so than fsc.

I looked at Ant and Maven, though haven't employed either yet (I'll also need to spend time solving #3 and #4). I wanted to see if anyone has other suggestions before I spend time getting a suboptimal build system working. Thanks in advance!

UPDATE - I'm now using Maven, passing a project as a compiler plugin to it. It seems fast enough; I'm not sure what kind of jar caching Maven does. A current repository for Scala 2.8.0 is available [link]. The archetypes are very cool, and cross-platform support seems very good. However, about compile issues, I'm not sure if fsc is actually fixed, or my project is stable enough (e.g. class names aren't changing) -- running it manually doesn't bother me as much. If you'd like to see an example, feel free to browse the pom.xml files I'm using [github].

UPDATE 2 - from benchmarks I've seen, Daniel Spiewak is right that buildr's faster than Maven (and, if one is doing incremental changes, Maven's 10 second latency gets annoying), so if one can craft a compatible build file, then it's probably worth it...

Answer 1

Have you looked at Intellij IDEA and its Scala integration ? Intellij has a loyal (fanatical?) following amongst Java developers, so you may find this is appropriate for your needs.

Answer 2

Points 2 and 4 are extremely difficult to manage with the current scalac. The problem is that Scala's compiler is a little dumb about building files. Basically, it will build whatever you feed it, regardless of whether or not that file really needs to be built. Scala 2.8.0 will have some tremendous improvements in this respect, but until then... Eclipse SDT actually has some very elaborate (and very hackish) code for doing change detection and dependency tracking. On the whole, it does a decent job, but as you have seen, there are wrinkles. Eclipse SDT 2.8.0 will rely on the aforementioned improvements to scalac itself.

So, building only modified files is pretty much out of the question. Aside from SDT, the only tool I know of which even tries this is SBT (Simple Build Tool). It uses a compiler plugin to track files as they are compiled and query the dependency graph computed by the compiler itself. In practice, this yields about a 50% improvement over the recompile-the-world approach. Once again, this is a hack to get around deficiencies in pre-2.8.0 scalac.

The good news is that reasonably fast compilation is still achievable even without worrying about change detection. FSC uses the same technology (ooh, that sounded so "Charlie Eppes") that Eclipse SDT uses to implement fast incremental compilation. In short, it's pretty snappy.

Personally, I use Apache Buildr. Its configuration is significantly cleaner than either Maven's or SBT's and its startup time is orders of magnitude less (when running under MRI). It integrates with FSC and attempts to do some basic change detection on its own (fairly primitive). It also has auto-magical support for the major Scala test frameworks (ScalaTest, ScalaCheck and Specs) as well as support for joint compilation with Java sources and IDE meta generation for IntelliJ and Eclipse. Oh, and it supports all of Maven's features (dependency resolution, etc) and then some. I'm even working on an extension which would allow interactive shell support integrated with JavaRebel and supporting several shell providers (Scala, JIRB, Clojure REPL, etc). It's not ready for the SVN yet, but I'll commit once it's ready (possibly in time for 1.3.5).

As you can see, I'm very firmly of the opinion that Buildr is the best Scala build tool out there. Its documentation is a little spotty where Scala is concerned, but that's because everything is so straightforward that it's hard to document without feeling verbose. You can always check out one of my GitHub repositories for examples. Good luck!

Answer 3

Am also quite frustrated with the scala plugin on Eclipse and I can add a few more problems to the list:

• auto-complete only works some of the time
• the debugger doesn't work properly (especially when trying to debug scala xml)
• the debugger forgets breakpoints
• 'go to definition' doesn't work more often than not.

I'm glad to hear that Buildr sounds like a better alternative (on the build front anyhow), I'll give that a try - thanks!

Answer 4

I went down the same road, and here is where I am at: - After some initial investigation, I dropped Kate. I love to use it for most things, but when it came to things like defining tab completions, I found it sorely lacking. I would recommend that you look into gedit instead, which is much more robust for Scala development - With gedit as my editor, I use SBT and have found it to be a great build tool. I can put it into a 'test' mode where when any code changes it recompiles the relevant files and runs my test suite. This has been an extremely effective way to work.

I have not taken a look at Buildr yet. I would like to say that I will, but honestly with SBT at my disposal I don't really have a compelling need to look at another build tool.

Answer 5

If you want to use Eclipse, but build the project using sbt, and still be able to debug, take a look at this post here:

zikaprog.wordpress.com/2010/04/19/scala-eclipse-sbt-and-debugging/

It also can be applied to builders other than sbt.

Answer 6

If you use Emacs, I think Ensime is a pretty good IDE. I think at the time writing, Ensime is the only IDE that will give you fast and accurate autocompletion on both Scala and Java objects, including implicit conversions.

There's code browsing support using Speedbar, code templates using the excellent Yasnippet, and code completion menu using Autocomplete. These are all very modern, actively maintained Emacs packages. There's also out of the box incremental building support for Maven and SBT.

There's a lot more in there such as interactive debugging, refactoring, and the Scala interpreter in an inferior process. All the things you want in a modern IDE for Scala is already there in Ensime. Highly recommended for Emacsens.