Will writing my Web appication in a functional programming language such as Haskell or Erlang make it more scalable than J2EE?

Question

Hi

I am writing a Facebook application that would use a Postgres DB along with Facebook APIs and run on Amazon EC2. (and I am hoping for heavy loads )

With Java, I know that DB would be my primary bottleneck and concurrency limitations of Tomcat would be the secondary bottleneck. I could alleviate DB issues with caching and concurrency issues with horizontal scaling . ( but this would add to my EC2 costs).

How would Erlang or Haskell help in this situation ? ( assuming I am able to master the learning curve).

Answer 1

There is no way that choosing a different language is going to significantly speed up an application over a compiled language such as Java when the bottlenecks are already defined as not being code related. Most functional languages have more overhead than Java and so there has to be a compelling reason to switch to one if you are already familiar with Java.

Answer 2

Two semi-answers:

Do you have users yet? No? Then use whatever will help you get the project off the ground quicker. You can always rewrite things later if you have to. "Too many users" is a problem that most people would like to have, but don't. If you have real reason to expect a large user base quickly (e.g., you run a popular blog and expect many of your readers to join immediately) it's justified to worry about this, otherwise you're borrowing trouble.

Are you sure you know where the bottlenecks will be? Scaling out like that raises concerns very different from performance in a smaller application. Make sure you really know what's broken before you start preemptively fixing things. The architecture of your application will probably be more important that what you build it with, anyway.

That said, either Erlang or Haskell would work if you want to do it that way, but probably won't make a huge difference for what you're asking. There's plenty of other reasons to prefer them to Java, though...

Answer 3

I'd take a look at http://www.highscalability.com and look at case studies of how to go about scaling your application to larger and larger loads. In particular search there for Brad Fitzpatrick's description of how he scaled LiveJournal and Danga Interactive (eg, this 2005 presentation).

Your intuition about the database being the first bottleneck and then the web server is probably correct, but of course you need to measure.

The major ways to scale your site will involve clustering and caching and database sharding and so on. The choice of programming language is secondary, and generally affects the raw performance on each box. See Henderson's Building Scalable Web Sites and Schlossnagle's Scalable Internet Architectures for other ideas and background in this area.

Having said that, a functional language may help to improve your overall scalability. Twitter used Scala to improve back end performance. Scala is a JVM language that combines object-oriented and functional styles, supports the Actors concurrency model, and runs at nearly the speed of Java (Martin Odersky, the creator of Scala, also wrote the current Sun Java compiler). So if you should run into a concurrency bottleneck you might want to sprinkle a bit of Scala in with your Java.