Should the JUnit message state the condition of success or failure?

Question

I can write an assertion message one of two ways. Stating success:

assertEquals( "objects should be identical", expected, actual );

Or stating the condition of being broken:

assertEquals( "objects aren't identical", expected, actual );

Is there a standard for this in JUnit specifically? If not, what are the arguments for each side?

P.S. I've seen articles on the web demonstrating both of these without explanation, so just saying "search Google" is not an answer!

[UPDATE]

Everyone is getting hung up on the fact that I used assertEquals and therefore the message is probably useless. But of course that's just because I wanted to illustrate the question simply.

So imagine instead it's:

assertTrue( ... big long multi-line expression ... );

Where a message is useful.

Answer 1

I rarely even bother with a message, at least for assertEquals. Any sensible test runner will explain that you were using assertEquals and the two things which were meant to be equal. Neither of your messages give more information than that.

I usually find that unit test failures are transient things - I'll rapidly find out what's wrong and fix it. The "finding out what's wrong" usually involves enough detail that a single message isn't going to make much difference. Consider "time saved by having a message" vs "time spent thinking of messages" :)

EDIT: Okay, one case where I might use a message: when there's a compact description in text which isn't obvious from the string representation of the object.

For example: "Expected date to be December 1st" when comparing dates stored as milliseconds.

I wouldn't worry about how you express it exactly though: just make sure it's obvious from the message which way you mean. Either "should be" or "wasn't" is fine - just "December 1st" wouldn't be obvious.

Answer 2

I don't think it matters at all - You already know that a failure happened, and therefore it doesn't matter if the message states what should have happened, or what shouldn't happen.

The goal of the message is to help you when it can, not to obtain some completeness.

Obviously, in the case of assertEquals this is less important, but the message is important in the case of general asserts. The message should help you obtain enough context to understand right away what exactly failed.

However, the amount of needed context (and thus the details in the message) should depend on how you get the report. For example, if you get it in Eclipse, you can easily go and interact and see what happened, so the message is less imporrtant. However, if you get your reports emailed to you (e.g., from a continuous build server) then you want the message to provide enough information so that you will have an idea of what is going on before you even go to the corresponding source code.

Answer 3

Vote me down too (like Jon), but the only time I've ever use a message like this (on assert equals) is when building a single test with a matrix of values and one of the test elements fails: I use the message to indicate which test case failed. Otherwise, the text is totally redundant.

Answer 4

From the javadocs of JUnit:

Asserts that two objects are equal. If they are not an AssertionFailedError is thrown with the given message.

According to the API, the message can be whatever you want. I would argue that the two options you have are both the same and both superfluous. The success or failure of the assert already provides all the information you are providing in the message.

It follows for me that you should have either nothing (there is an assert that doesn't take a string on purpose) OR include a message with meaning beyond what is already there.

So I guess this is a reiteration of Jon's answer, but too verbose to be a comment.

Answer 5

I don't put a message for the case you cite, unless I'm running a test where I have an array of similar test values that I'm running in a loop and I want to pinpoint exactly which one failed. Then I add a message to tell me which one.

Answer 6

I would like to answer the question without considering, if a message in generel is useful.

If a test fails, something is wrong. I know this. I want to know why it is broken. That's very easy to find out because I just have to open the test case and the SUT. Like Jon said, it's very easy to fix it (hopefully ;-) ).

But what about the message? The message is for me an advice, what could be done to turn it into a green test case. So I would appreciate if there's an advice given in the message text, how to fix this problem or where to search for the problem.

Another interesting aspect would be the usage of positive expressions. It's worth a consideration to use positive text messages. In your example, I would use Objects should be identical. But that's a small reason.