Weird problem using JUnit in multi-thread environment

Question

Hi all,

I meet a weired problem when using JUnit in multi-thread environment. The following code should fail, but it actually pass in eclipse.

public class ExampleTest extends TestCase {

    private ExecutorService executor = Executors.newFixedThreadPool(10);

    private volatile boolean isDone = false;

    public void test() throws InterruptedException, ExecutionException {
        executor.submit(new Runnable() {

            @Override
            public void run() {
                try {
                    fail();
                } finally {
                    isDone = true;
                }
            }
        });

        while (!isDone) {
            Thread.sleep(1000);
        }
    }
}

And here'a another piece of code, here I use Future.get() to wait for thread stop, in this case it will fail.

public class ExampleTest extends TestCase {

    private ExecutorService executor = Executors.newFixedThreadPool(10);

    private volatile boolean isDone = false;

    public void test() throws InterruptedException, ExecutionException {
        Future future=executor.submit(new Runnable() {

            @Override
            public void run() {
                try {
                    fail();
                } finally {
                    isDone = true;
                }
            }
        });

        future.get();
    }
}

I googled it and found that JUnit can not handle Multiple-thread unit testing,but what's the differences between these two pieces of code ? Thanks

Answer 1

Take a look at http://www.youtube.com/watch?v=wDN_EYUvUq0 (starting at 17:09), it explain problems you can get with JUnit and threads.

I think, that in your case, get() throws a ExecutionException and that's why the second test fails. In the first testcase, jUnit doesn't see the exception.

Answer 2

JUnit cannot see the exceptions that occur in threads other than the thread in which the tests are running. In the first case, through an exception occurs by calling fail, it occurs in a separate thread run by the executor. Hence it is not visible to JUnit and the test passes.

In the second case, the same exception happens in the separate thread run by the executor but the exception is effectively "reported back" to the test thread when you call future.get. This is because future.get throws an ExecutionException if the computation of the future failed due to any exception. JUnit is able to see this exception and hence the test fails.

Answer 3

There is also the interesting fact that Eclipse and IDEA can spawn a VM in their junit test runners and end up calling system.exit() on it. This means if you don't wait properly in the test (as in the case when you sleep above and hope the the task has completed), it can exit unexpectedly. Interesting, but not exactly what you were asking!

see this link for details...

Answer 4

As @abhin4v has pointed out, the exception in the new thread gets swallowed. You could try providing your own fail-method that syncronises with the top-level thread very much like in your example with get().

But there's no need to use Futures, just write to a shared variable indicating failure and use newThreadId.join(). Apart from that, I'm not aware of any other way of solving this in plain JUnit.