How to run concurrency unit test?

Question

Hi,

How to use junit to run concurrency test?

Let's say I have a class

public class MessageBoard
{
    public synchronized void postMessage(String message)
    {
     ....
    }

    public void updateMessage(Long id, String message)
    {
     ....
    }
}

I wan to test multiple access to this postMessage concurrently. Any advice on this? I wish to run this kind of concurrency test against all my setter functions (or any methodn that involves create/update/delete operation).

Thanks

Answer 1

Unfortunately I don't believe you can definitively prove that your code is thread-safe by using run-time testing. You can throw as many threads as you like against it, and it may/may not pass depending on the scheduling.

Perhaps you should look at some static analysis tools, such as PMD, that can determine how you're using synchronisation and identify usage problems.

Answer 2

You can only prove the presence of concurrent bugs, not their absence.

However you can write a specialized test runner that spawns several concurrent thread and then calls your @Test annotated methods.

Answer 3

TestNG has support for concurrency testing in Java. This article describes how it can be used and there is docs on the testng site.

Not sure if you can make the same test run at the same time though

Answer 4

The best approach here is to use system tests to blast your code with requests to see if it falls over. Then use the unit test to check the logical correctness. The way i would approach this is create a proxy for the asynchronous call and have it made synchronously under test.

However, if you wanted to do this at a level other than a full install and complete environment you could do this in junit by creating your object in a separate thread, then create lots of threads that fire requests to your object and block the main thread until it completes. This approach can make tests fail intermittently if you don't get it right.

Answer 5

In .NET, there are tools like TypeMock Racer or Microsoft CHESS that are designed specifically for unit testing concurrency. These tools not only find multithreading bugs like deadlocks, but also give you the set of thread interleaves that reproduce the errors.

I'd imagine there's something similar for the Java world.

Answer 6

In your example the postMessage() method is synchronized, so you won't actually see any concurrency effects from within a single VM but you might be able to evaluate the performance of the synchronized version.

You will need to run multiple copies of the test program at the same time in different VMs. You can use

If you can't get your test framework to do it you can launch some VMs your self. The process builder stuff is a pain with paths and whatnot, but here is the general sketch:

Process running[] = new Process[5];
for (int i = 0; i < 5; i++) {
 ProcessBuilder b = new ProcessBuilder("java -cp " + getCP() + " MyTestRunner");
 running[i] = b.start();
}

for(int i = 0; i < 5; i++) {
 running[i].waitFor();
}

I usually do something like this for simple threaded tests, like others have posted, testing is not a proof of correctness, but it usually shakes out silly bugs in practice. It helps to test for a long time under a variety of different conditions -- sometimes concurrency bugs take a while to manifest in a test.

public void testMesageBoard() {
 final MessageBoard b = new MessageBoard();

 int n = 5;
 Thread T[] = new Thread[n];
 for (int i = 0; i < n; i++) {
  T[i] = new Thread(new Runnable() {
   public void run() {
    for (int j = 0; j < maxIterations; j++) {
      Thread.sleep( random.nextInt(50) );
      b.postMessage(generateMessage(j));
      verifyContent(j); // put some assertions here
    }
   }
  });

  PerfTimer.start();
  for (Thread t : T) {
   t.start();
  }

  for (Thread t : T) {
   t.join();
  }
  PerfTimer.stop();
  log("took: " + PerfTimer.elapsed());
 }
}**strong text**

Answer 7

Testing concurrency bugs is impossible; you don't just have to validate input/output pairs, but you have to validate state in situations which may or may not occur during your tests. Unfortunately JUnit is not equipped to do this.

Answer 8

I would recommend using MultithreadedTC - Written by the concurrency master himself Bill Pugh (and Nat Ayewah). Quote from their overview:

MultithreadedTC is a framework for testing concurrent applications. It features a metronome that is used to provide fine control over the sequence of activities in multiple threads.

This framework allows you to deterministically test every thread interleaving in separate tests

Answer 9

You can use the tempus-fugit library to run test methods in parallel and multiple times to simulate a load testing type environment. Although a previous comment points out that the post method is syncrhonised and so protected, associated members or methods may be involved which themselves are not protected, so its possible that a load / soak type test could catch these. I'd suggest you setup a fairly cause grained / end-to-end like test to give you the best chance of catching any loop holes.

See http://tempus-fugit.googlecode.com/svn/site/documentation/concurrency.html#JUnit_Integration

Answer 10

Try looking at ActiveTestSuite which ships with JUnit. It can concurrently launch multiple JUnit tests:

public static Test suite()
{
    TestSuite suite = new ActiveTestSuite();
    suite.addTestSuite(PostMessageTest.class);
    suite.addTestSuite(PostMessageTest.class);
    suite.addTestSuite(PostMessageTest.class);
    suite.addTestSuite(PostMessageTest.class);
    suite.addTestSuite(PostMessageTest.class);
    return suite;
}

The above will run the same JUnit test class 5 times in paralell. If you wanted variation in your paralell tests, just create a different class.