Is there a multi-process unit testing framework / junit addon?

Question

Imagine two servers (each within an own jvm process) which communicate using some form of messages (e.g. simple producer/consumer)

I'd like to write unit tests that will test the behaviour of these two servers.
My questions are:

1. Are there some frameworks (or a junit addon) for this problem?
   I'd like to run a junit test class (or even a single test) in a different process? It would be cool if one unit test can easily access the variables of the other test (in a different process) using some sort of inter-process communication.
   E.g. I'd like to check if the producer really produced the things i expected, and afterwards, if the consumer consumed them. (and maybe do some fuzzing to detect some race-condition-related problems)

2. Are there some best practice for this kind of testing?

3. If there isn't a unit-like testing approach, how would you test such things during continous integration?

Note:
I don't want to use threads, because this may alter the behaviour (if you think of thread-safety issues)

Answer 1

I'd recommend using mock objects, you can then test the producer and the consumer independently mocking out the other. So you test the producer produces, and the consumer consumes.

You can also test the mechanism the two communicate, but I'd expect thats provided by a 3rd party?

See Mockito, jMock, EasyMock

Answer 2

What you are talking in point 1 is not a true unit testing scenario. In a perfect unit test you will not worry about who is producing the messages and who is consuming the messages. One of your test case will concentrate on imagining(or mocking) that you received different messages from a valid producer and your test cases will test how these are being consumed properly. What you are looking is more like a integration testing. Anyway some of the statements which I made may be subjective. You can use jMock, easymock frameworks for your mocking needs.

Answer 3

This is way beyond Unit testing and firmly inside Integration testing.

I would recommend to test what you can by mocking out your comms layer with each piece separate.

What I did in the past in such cases is to start a separate thread in the test which starts a Mock Receiver/Sender. In the main test I then do the Sender/Receiver part. In practice this means this is full of delays to make sure things start in the right order and this becomes dead slow, so you just want to do this to test that the pieces of the puzzle fit, and do as little as possible functional testing over it.

I verify the desired behavior and terminate the helper thread before leaving the test.

This can test a lot. Testing using separate processes (and hosts!) is a real pain. takes forever and I would limit that to manual testing.

Answer 4

I would describe what you are looking to do as an integration test, and beyond the scope of JUnit as a framework (JUnit is only now dipping its toe into multi-thread testing, multi-process is certainly not in the feature set).

You could of course use JUnit to run such tests, but they would really be limited to a runner, the rest you have to do yourself.

Others have already pointed to the fact that you would usually mock or otherwise send artificially constructed methods to the consumer, and test what the producer produces independently at the "unit test" level.

In terms of actually testing the interaction between the producer and the consumer, one approach is to forget about the inter-process testing, and test it intra-process on one thread via some sort of Dependency Injection where the producer sends the message via some fake way that just passes it on the the consumer without anything more under the hood than in-thread method calls.

However, it seems that you want to test things that can happen with the actual inter-process stuff on top of it (race conditions and the like), which makes this all-the-more an integration test.

To approach this problem, you need to start the process and wait for it to accept a message, then your test would tell the producer what message to create and it would send it to the consumer. Then your test would ask the consumer what it got (with suitable delay).

The need here would have to be compelling for me to do it. I would go for full blown automated acceptance testing and let that encompass this level of integration.