Any satisfactory approaches to unit testing thread safety in Java?

Question

I am looking at improving a package that I believe not to be threadsafe when its input is shared between multiple worker threads. According to TDD principles, I should write some tests that fail in the first instance, and these would certainly be useful in assessing the problem.

I realise that this is not a simple thing to acheive, and that naively, multi-threaded tests will be nondeterministic as the operating system will determine scheduling and the exact order that various operations are interleaved. I have looked at and used MultithreadedTC in the past, and this was useful. However, in that case I knew in advance exactly where the existing implementation fell down, and thus was able to cook up a nice set of tests that covered it.

However, if you're not at the point where you know exactly what the problem is, is there a good way of going about writing a test that stands a good chance of throwing up any potential problems? Are there any libraries that others have found helpful? Would I be right in thinking that from a purist point of view, a multi-threaded test case should just be the same calls and assertions as the usual single-threaded test, only run with multiple worker threads as appropriate?

Any offers on tools/best practices/philosophy in general would be welcome.

Answer 1

Forget getting good results by testing for concurrency problems. Attempt to reduce synchronization and make the problem smaller. Then use as high as possible library support to do synchronization. And only if you really have then try to handle the concurrency yourself. When you know each worker does its work correct and all your thinking tell you that you have the concurrency problem licked then generate some interesting load. Unittest frameworks and their extensions can do that job, but know that you are not testing any units anymore. (Remember, you already had that part covered)

If your concurrency model gets complicated check out tools suited for that like SPIN.

Answer 2

In some cases, I've found I can force a particular problematic interleaving of calls to a potentially un-thread-safe class by using multiple threads that synchronize with each other, perhaps with the use of CountDownLatch or some such concurrency mechanism. Sometimes this just doesn't work however, for instance if you're trying to test what happens if two threads are in the same method at the same time.

Here's an interesting article (don't know much about the tool, though) : http://today.java.net/pub/a/today/2003/08/06/multithreadedTests.html

Answer 3

Java Concurrency in Practice has some great information about how to write tests for concurrency issues. However they are not true unit tests. It is nearly impossible to write a true unit test for a concurrency issue.

Basically it boils down to this. Create a bunch of test threads and start them. Each thread should

• wait for a count down latch
• repeatedly call some method that modifies the mutable state in question
• count down on a second latch and exit

The junit thread creates all the threads and starts them, then counts down on the first latch once to let them all go, then waits for the second latch, then makes some assertions about the mutable state.

Even more so than other types of bugs, it's easier to write a failing unit test for a concurrency bug after you have found the bug.