how much concurrent http request can erlang handle

Question

I am developing a application for benchmarking purposes, for which I require to create large number of http connection in a short time, I created a program in java to test how much threads is java able to create, it turns out in my 2GB single core machine, the limit is variable between 5000 and 6000 with 1 GB of memory given to JVM after which it hits outofmemoryerror with heap limit reached.

It is suggested that erlang will be able to generate much more concurrent processes, I am willing to learn erlang if it is capable of solving the problem , can erlang be able to generate somewhere around 100000 processes which are essentially http requests waiting for responses, in a matter of few seconds without reaching any limit like memory error etc.,

Answer 1

It can handle pretty much anything you throw at it. Erlang processes are extremely light weight.

See http://www.sics.se/~joe/apachevsyaws.html for a benchmark in concurrency between Yaws and Apache. It gives you a good idea of what's possible.

Answer 2

According famous Richard Jones blog you can handle 100k connection almost out of the box. You have to increase process limit, see +P parameter and it needs little bit memory management trickery e.g. gc or hibernate. To achieve significantly more you have to do more hacking with C.

Answer 3

I thought an interesting thing for this is that a guy was able to get a million comet connections with mochiweb http://www.metabrew.com/article/a-million-user-comet-application-with-mochiweb-part-1

Answer 4

As previously stated, a lot is a good answer. And also given your requirement is to write a tsunami client, you can/should distribute code over to several erlang nodes.

Even better, you might want to check out Tsung. It is a distributed load testing application written in erlang.

And if you don't want to use it, I am pretty sure there's code you want to read in there.