Unit Testing without Assertions

Question

Occasionally I come accross a unit test that doesn't Assert anything. The particular example I came across this morning was testing that a log file got written to when a condition was met. The assumption was that if no error was thrown the test passed.

I personally don't have a problem with this, however it seems to be a bit of a "code smell" to write a unit test that doesn't have any assertions associated with it.

Just wondering what people's views on this are?

Answer 1

In some sense, you are making an implicit assertion - that the code doesn't throw an exception. Of course it would be more valuable to actually grab the file and find the appropriate line, but I suppose something's better than nothing.

Answer 2

In general, I see this occuring in integration testing, just the fact that something succeeded to completion is good enough. In this case Im cool with that.

I guess if I saw it over and over again in unit tests I would be curious as to how useful the tests really were.

EDIT: In the example given by the OP, there is some testable outcome (logfile result), so assuming that if no error was thrown that it worked is lazy.

Answer 3

It's simply a very minimal test, and should be documented as such. It only verifies that it doesn't explode when run. The worst part about tests like this is that they present a false sense of security. Your code coverage will go up, but it's illusory. Very bad odor.

Answer 4

It can be a good pragmatic solution, especially if the alternative is no test at all.

The problem is that the test would pass if all the functions called were no-ops. But sometimes it just isn't feasible to verify the side effects are what you expected. In the ideal world there would be enough time to write the checks for every test ... but I don't live there.

The other place I've used this pattern is for embedding some performance tests in with unit tests because that was an easy way to get them run every build. The tests don't assert anything, but measure how long the test took and log that.

Answer 5

I have to admit that I have never written a unit test that verified I was logging correctly. But I did think about it and came across this discussion of how it could be done with JUnit and Log4J. Its not too pretty but it looks like it would work.

Answer 6

We actually have a way to document such tests in xUnit.net. You can either do:

Assert.DoesNotThrow(() => someCode());

Or if you prefer something more in the Arrange/Act/Assert style:

// Act
Exception ex = Record.Exception(() => someCode());

// Assert
Assert.Null(ex);

Answer 7

Such a test smells. It should check that the file was written to, at least that the modified time was updated perhaps.

I've seen quite a few tests written this way that ended up not testing anything at all i.e. the code didn't work, but it didn't blow up either.

If you have some explicit requirement that the code under test doesn't throw an exception and you want to explicitly call out this fact (tests as requirements docs) then I would do something like this:

try
{
  unitUnderTest.DoWork()
}
catch
{
  Assert.Fail("code should never throw exceptions but failed with ...")
}

... but this still smells a bit to me, probably because it's trying to prove a negative.

Answer 8

Tests should always assert something, otherwise what are you proving and how can you consistently reproduce evidence that your code works?

Answer 9

We do this all the time. We mock our dependencies using JMock, so I guess in a sense the JMock framework is doing the assertion for us... but it goes something like this. We have a controller that we want to test:

Class Controller {
  private Validator validator;

  public void control(){
    validator.validate;
  }

  public setValidator(Validator validator){ this.validator = validator; }
}

Now, when we test Controller we dont' want to test Validator because it has it's own tests. so we have a test with JMock just to make sure we call validate:

public void testControlShouldCallValidate(){
  mockValidator.expects(once()).method("validate");
  controller.control;
}

And that's all, there is no "assertion" to see but when you call control and the "validate" method is not called then the JMock framework throws you an exception (something like "expected method not invoked" or something).

We have those all over the place. It's a little backwards since you basically setup your assertion THEN make the call to the tested method.

Answer 10

I've seen something like this before and I think this was done just to prop up code coverage numbers. It's probably not really testing code behaviour. In any case, I agree that it (the intention) should be documented in the test for clarity.

Answer 11

These are known as smoke tests and are common. They're basic sanity checks. But they shouldn't be the only kinds of tests you have. You'd still need some kind of verification in another test.

Answer 12

If there is no assertion, it isn't a test.

Quit being lazy -- it may take a little time to figure out how to get the assertion in there, but well worth it to know that it did what you expected it to do.

Answer 13

I sometimes use my unit testing framework of choice (NUnit) to build methods that act as entry points into specific parts of my code. These methods are useful for profiling performance, memory consumption and resource consumption of a subset of the code.

These methods are definitely not unit tests (even though they're marked with the [Test] attribute) and are always flagged to be ignored and explicitly documented when they're checked into source control.

I also occasionally use these methods as entry points for the Visual Studio debugger. I use Resharper to step directly into the test and then into the code that I want to debug. These methods either don't make it as far as source control, or they acquire their very own asserts.

My "real" unit tests are built during normal TDD cycles, and they always assert something, although not always directly - sometimes the assertions are part of the mocking framework, and sometimes I'm able to refactor similar assertions into a single method. The names of those refactored methods always start with the prefix "Assert" to make it obvious to me.

Answer 14

The name of the test should document this.

void TestLogDoesNotThrowException(void) {
    log("blah blah");
}

How does the test verify if the log is written without assertion ?