What's the best way to unit test concurrent Erlang code?

Question

I'm spending a bit of time with Erlang, and I'm wanting to apply TDD to code I'm writing.

While EUnit in the standard lib provides a nice traditional unit testing framework for testing regular style code, there doesn't seem to be anything to help specifically with testing concurrent code, which is used a LOT in Erlang.

Note that we're talking Erlang here, which uses message passing (as opposed to shared state) for communication between concurrent processes, so techniques for unit testing concurrent code in shared state languages may not be applicable.

Anyone found a good way to test concurrent code in Erlang?

Answer 1

Because errors in concurrent code manifest themselves depending on the order various parts get executed, I think it's best not to rely on testing to uncover your bugs in the concurrent part of your code. Such bugs are very easy to slip through and extremely difficult to locate. See for instance how the double-locking technique, which was widely cited and used as an efficient method for implementing lazy initialization in a multithreaded environment, was later found to be broken. It's far better to use appropriate abstractions and techniques to make the concurrent parts of your code correct "by design".

Answer 2

Adding to what Diomidis said, the only real way to gain some confidence with your concurrent code is to run extended tests under constantly varying conditions; and even then, it's only proof that it didn't fail under those conditions.

Answer 3

The one tool I know of that you can use to test Erlang programs under concurrent scenarios is QuickCheck: http://www.quviq.com/

You can specify your expected behaviour using properties, and the tool can exercise your Erlang program over a range of inputs and time to ensure the properties are satisfied.

Unfortunately it's a commercial tool

Also have a look at the ProTest project: http://www.protest-project.eu

Update: The ProTest project have released a survey, "Results for: Erlang testing tools survey for the ProTest project"

Answer 4

You could isolate out the part you would like to test with mocks by using this: http://github.com/charpi/erl_mock/tree/master

I have no experience with erl_mock though.

Answer 5

There is an early version of McErlang available, a model checker for Erlang:

The ambition is to support a large part of the rather complex Erlang API (module erlang:), including full support for processes, communication, distribution through nodes, process links, monitors, and so on. The current version of the tool is a prototype that partly implements this API, lacking for instance support of the Erlang port concept (to communicate with the outside world). Moreover the tool comes with tailored versions of the two most heavily used OTP library components, the generic server and the supervisor.