Assertion messages: assume failure, or assume success

Question

When testing in any language, how does everybody phrase their assertion messages?

I see three obvious ways:

# assume failure
assert (4-2) == 2, "Subtracting 2 from 4 doesn't equal 2"

# describe success
assert (4-2) == 2, "Subtracting 2 from 4 should equal 2"

# be vauge with failure
assert (4-2) == 2, "Subtracting 2 from 4 is broken"

This is obviously a simple example, but you get the idea. What is the standard practice? What do you do? Why?

Answer 1

The important thing with an assert is the actual condition being tested. In C you can use the preprocessor "stringization" to output the actual condition being tested. My code just outputs

Assert Failed: (4-2)==2 : Line 123, File foo.c

If you're lucky, you can get a stack dump as well...

Answer 2

It really doesn't matter IMO, as long as the error message tells you what went wrong and where the problem is. Under normal operation, an assertion message will never be seen. If it is seen, there's a bug in the code somewhere, and you need to be able to track down and fix the bug.

The message should provide as much helpful information as possible - if you're checking that two values are equal and they're not, you should print out what the values are. That obviates the need to step into it with a debugger to examine the values. Of course, doing this requires that your language supports non-static information in assertion messages. If assertion messages can only be fixed, static strings, you can't add extra runtime information without jumping through hoops.

Answer 3

I don't know what's the standard practice, but I combine your first two approaches with the addition of the actual result obtained.

"Substracting 2 from 4 should equal 2, but equals " + value

This leaves no room for doubt and eases debugging.

Answer 4

Roddy's suggestion is a good one--it certainly makes the "cost" of adding new assertions much cheaper (i.e. doesn't require thinking of or typing out a new string message). Believe it or not, but implementing his suggestion has had a significant impact on my usage of assertions.

If you're still looking for a clearer text message, though, you might consider something like:

"Expected 4-2 to equal 2"

This seems to make clear (at least to me) that the expected answer was 2, but that the expectation was not met...

Answer 5

I don't know if there's any "standard", but in most cases I like to see the assertion condition itself. C does this automatically. In C# I have to copy-paste it, unfortunately, eg.

Debug.Assert(4-2==2, "4-2==2");

In some cases it's useful to see more information than the condition provides. In that case I treat the assert message like an error message and state what's wrong, eg.

Debug.Assert(result != null, "No result returned for input '" + input + "'");

Answer 6

I use the assertion message to document what the assertion is testing, or what the expected value is about.

Sometimes, when a test fails, I do not need to look at the failing test : just from assertion message and the change(s) I just made, I know how to fix it.

Answer 7

I treat assertion messages like comments: I try not to have them if I can avoid it, if I can make the intent clear from the code.

Usually this is an issue when there are several attributes you want to assert on that are related in some way. Extracting the assertions into a method named to indicate the aspect/relationship you're interested in makes it more obvious what is going on without the cost of maintaining the comment/message.