Erlang - C and Erlang

Question

I am building a program that mainly parses binary data and does some number crunching on it.
It's primary goal is to do it all as quickly as possible. It will be running on an 8 - 32 core machine [Xeon / Sparc T2]. I choose erlang becasue I beleive that concurrency is much easier as opposed to c++.

However, about halfway through I start seeing more and more "benchmarks" that show c would be faster. I am not willing to fully switch to c, becasue I believe the easy concurrency provided by erlang would outweigh the speed at which c can do it AND the amount of code needed for concurrency in c.

So my question is would it be ideal to have c code do the binary parsing and number crunching, and have erlang spawn process' to run the c code.

If so can anyone point me to good book (doubt there is one) or tutorial on c and erlang in harmony.. What a beautiful name for a book

Answer 1

If you're unexperienced, these decisions are hard to make up front. A viable strategy would be to build a prototype in a language you're most familiar with, and later optimize parts as required. For example, you can continue with Erlang and if eventually you find (by profiling!) that the binary parsing & number crunching code is the bottleneck in terms of speed, rewrite those parts in C.

Answer 2

First of all write whole logic of the system in Erlang, then implement handling binaries in C. Using NIFs (it is kind of interface to C) is pretty straight forward and transparent to the rest of the system. Here is another thread about talking to C http://stackoverflow.com/questions/3525457/run-c-code-block-in-erlang.

Before hacking C, make sure you benchmarked current implementation. It is possible that it will satisfy your needs especially with the latest Erlang/OTP release (R14) which introduces great enhancements to binary handling.

Answer 3

easy threading is not so interesting to erlang. Easy threading + Message passing and the OTP framework is what's awesome about erlang. If you need number crunching use something like ocaml, python, haskell. Erlang is all that good at number crunching.

Parsing binaries is one of the things erlang is best at though, probably the best for it. Joe's book programming erlang covers everything really well, and is not so expensive used. It also talks about integrating C code and gives an example. the source is available from pragmatic programming without needing to buy the book, you can grep #include or something.

Answer 4

Of course C would be faster in he extreme case after optimisations. If by faster you mean faster to run.

Faster to write would be Erlang by far. Depending on the speed requirements you have Erlang is probably "fast enough", and it will save you days of searching for bugs in C.

And the C code will only be faster after optimisations. If you spend the same amount of time on C and Erlang you will come out with about the same speed (note that I count time spent debugging and error fixing in this time estimation. Which will be a lot less in Erlang).

So:

faster writing = Erlang
faster running (after optimisations) = C
faster running without optimisations = any of the two

Take your pick.

Answer 5

If you like Erlang, but want C speed, why not go for JOCAML. It is an extension for OCAML (which is similar to Erlang but is near C in terms of speed) designed for the multicore revolution going on at the moment. I love it (and I know more than 10 programming languages...)