Why does IntelliJ IDEA compile Scala so slowly?

Question

I am using idea 9.0.3 which is the latest release. When I write some scala source code in Idea, it takes several seconds to compile and run. This shouldn't be that slow, Is this normal?

Answer 1

Yes, this is normal for IDEA. Eclispe is faster. I recommend the latest nightly build of the eclipse plugin for eclipse 3.6

Answer 2

I have experienced that scala generally takes a long time to compile, also when doing it from the command line.

Answer 3

Idea can be slower in a small project but it can compile huge java and scala mixed project in reasonable time while Eclipse just fails at all.

You might also want to try fsc with idea: http://devnet.jetbrains.net/thread/286420

Answer 4

Are you comparing the compilation of Scala with IDEA to:

• Scala via scalac at the command line
• Scala with Eclipse?
• Java with IDEA?

If it's the latter, then you should be aware that Scala compilation takes at least an order of magnitude longer than Java to compile. The reasons for this are as follows:

1. Naming conventions (a file XY.scala file need not contain a class called XY and may contain multiple top-level classes). The compiler may therefore have to search more source files to find a given class/trait/object identifier.
2. Implicits - heavy use of implicits means the compiler needs to search any in-scope implicit conversion for a given method and rank them to find the "right" one. (i.e. the compiler has a massively-increased search domain when locating a method.)
3. The type system - the scala type system is way more complicated than Java's and hence takes more CPU time.
4. Type inference - type inference is computationally expensive and a job that javac does not need to do at all
5. scalac includes an 8-bit simulator of a fully armed and operational battle station, viewable using the magic key combination CTRL-ALT-F12 during the GenICode compilation phase.

If it's either of the former, then you may be right and your IDEA configuration may not be optimal. Have to tried the fsc (i.e. fast compilation server) integration in the latest builds? Basically fsc is a permanently-running compilation server, which removes the overhead of both JVM start-up time and the lack of JIT-optimization in short-running processes.

Answer 5

The best way to do Scala is with IDEA and SBT. Set up an elementary SBT project (which it'll do for you, if you like) and run it in automatic compile mode (command ~compile) and when you save your project, SBT will recompile it.

You can also use the SBT plug-in for IDEA and attach an SBT action to each of your Run Configurations. The SBT plug-in also gives you an interactive SBT console within IDEA.

Either way (SBT running externally or SBT plug-in), SBT stays running and thus all the classes used in building your project get "warmed up" and JIT-ed and the start-up overhead is eliminated. Additionally, SBT compiles only source files that need it. It is by far the most efficient way to build Scala programs.

Randall Schulz

Answer 6

There are two aspects to the (lack of) speed for the Scala compiler.

1. Greater startup overhead

   • Scalac itself consists of a LOT of classes which have to be loaded and jit-compiled

   • Scalac has to search the classpath for all root packages and files. Depending on the size of your classpath this can take one to three extra seconds.

   Overall, expect a startup overhead of scalac of 4-8 seconds, longer if you run it the first time so disk-caches are not filled.

   Scala's answer to startup overhead is to either use fsc or to do continuous building with sbt. IntelliJ needs to be configured to use either option, otherwise its overhead even for small files is unreasonably large.

2. Slower compilation speed. Scalac manages about 500 up to 1000 lines/sec. Javac manages about 10 times that. There are several reasons for this.

   • Type inference is costly, in particular if it involves implicit search.

   • Scalac has to do type checking twice; once according to Scala's rules and a second time after erasure according to Java's rules.

   • Besides type checking there are about 15 transformation steps to go from Scala to Java, which all take time.

   • Scala typically generates many more classes per given file size than Java, in particular if functional idioms are heavily used. Bytecode generation and class writing takes time.

   On the other hand, a 1000 line Scala program might correspond to a 2-3K line Java program, so some of the slower speed when counted in lines per second has to balanced against more functionality per line.

   We are working on speed improvements (for instance by generating class files in parallel), but one cannot expect miracles on this front. Scalac will never be as fast as javac. I believe the solution will lie in compile servers like fsc in conjunction with good dependency analysis so that only the minimal set of files has to be recompiled. We are working on that, too.