design by contract tests by assert or by exception?

Question

When programming by contract a function or method first checks whether its preconditions are fulfilled, before starting to work on its responsibilities, right? The two most prominent ways to do these checks are by assert and by exception.

1. assert fails only in debug mode. To make sure it is crucial to (unit) test all separate contract preconditions to see whether they actually fail.
2. exception fails in debug and release mode. This has the benefit that tested debug behavior is identical to release behavior, but it incurs a runtime performance penalty.

Which one do you think is preferable?

See releated question here

Answer 1

Disabling assert in release builds is like saying "I will never have any issues whatsoever in a release build", which is often not the case. So assert shouldn't be disabled in a release build. But you don't want the release build crashing whenever errors occur either, do you?

So use exceptions and use them well. Use a good, solid exception hierarchy and ensure that you catch and you can put a hook on exception throwing in your debugger to catch it, and in release mode you can compensate for the error rather than a straight-up crash. It's the safer way to go.

Answer 2

you're asking about the difference between design-time and run-time errors.

asserts are 'hey programmer, this is broken' notifications, they're there to remind you of bugs you wouldn't have noticed when they happened.

exceptions are 'hey user, somethings gone wrong' notifications (obviously you can code to catch them so the user never gets told) but these are designed to occur at run time when Joe user is using the app.

So, if you think you can get all your bugs out, use exceptions only. If you think you can't..... use exceptions. You can still use debug asserts to make the number of exceptions less of course.

Don't forget that many of the preconditions will be user-supplied data, so you will need a good way of informing the user his data was no good. To do that, you'll often need to return error data down the call stack to the bits he is interacting with. Asserts will not be useful then - doubly so if your app is n-tier.

Lastly, I'd use neither - error codes are far superior for errors you think will occur regularly. :)

Answer 3

The rule of thumb is that you should use assertions when you are trying to catch your own errors, and exceptions when trying to catch other people's errors. In other words, you should use exceptions to check the preconditions for the public API functions, and whenever you get any data that are external to your system. You should use asserts for the functions or data that are internal to your system.

Answer 4

I prefer the second one. While your tests may have run fine, Murphy says that something unexpected will go wrong. So, instead of getting an exception at the actual erroneous method call, you end up tracing out a NullPointerException (or equivalent) 10 stack frames deeper.

Answer 5

The principle I follow is this: If a situation can be realistically avoided by coding then use an assertion. Otherwise use an exception.

Assertions are for ensuring that the Contract is being adhered to. The contract must be fair, so that client must be in a position to ensure it complies. For example, you can state in a contract that a URL must be valid because the rules about what is and isn't a valid URL are known and consistent.

Exceptions are for situations that are outside the control of both the client and the server. An exception means that something has gone wrong, and there's nothing that could have been done to avoid it. For example, network connectivity is outside the applications control so there is nothing that can be done to avoid a network error.

I'd like to add that the Assertion / Exception distinction isn't really the best way to think about it. What you really want to be thinking about is the contract and how it can be enforced. In my URL example above that best thing to do is have a class that encapsulates a URL and is either Null or a valid URL. It is the conversion of a string into a URL that enforces the contract, and an exception is thrown if it is invalid. A method with a URL parameter is much clearer that a method with a String parameter and an assertion that specifies a URL.

Answer 6

See also this question

Answer 7

The previous answers are correct: use exceptions for public API functions. The only time you might wish to bend this rule is when the check is computationally expensive. In that case, you can put it in an assert.

If you think violation of that precondition is likely, keep it as an exception, or refactor the precondition away.

Answer 8

You should use both. Asserts are for your convenience as a developer. Exceptions catch things you missed or didn't expect during runtime.

I've grown fond of glib's error reporting functions instead of plain old asserts. They behave like assert statements but instead of halting the program, they just return a value and let the program continue. It works surprisingly well, and as a bonus you get to see what happens to the rest of your program when a function doesn't return "what it's supposed to". If it crashes, you know that your error checking is lax somewhere else down the road.

In my last project, I used these style of functions to implement precondition checking, and if one of them failed, I would print a stack trace to the log file but keep on running. Saved me tons of debugging time when other people would encounter a problem when running my debug build.

#ifdef DEBUG
#define RETURN_IF_FAIL(expr)      do {       \
 if (!(expr))                                           \
 {                                                      \
  fprintf(stderr,          \
  "file %s: line %d (%s): precondition `%s' failed.", \
  __FILE__,                                           \
  __LINE__,                                           \
  __PRETTY_FUNCTION__,                                \
  #expr);                                             \
  ::print_stack_trace(2);                             \
  return;                                                \
 };               } while(0)
#define RETURN_VAL_IF_FAIL(expr, val)  do {                \
 if (!(expr))                                                 \
 {                                                            \
 fprintf(stderr,                                 \
  "file %s: line %d (%s): precondition `%s' failed.",     \
  __FILE__,                                               \
  __LINE__,                                               \
  __PRETTY_FUNCTION__,                                    \
  #expr);                                                 \
  ::print_stack_trace(2);                                 \
  return val;                                                \
 };               } while(0)
#else
#define RETURN_IF_FAIL(expr)
#define RETURN_VAL_IF_FAIL(expr, val)
#endif

If I needed runtime checking of arguments, I'd do this:

char *doSomething(char *ptr)
{
    RETURN_VAL_IF_FAIL(ptr != NULL, NULL);  // same as assert(ptr != NULL), but returns NULL if it fails.
                                            // Goes away when debug off.

    if( ptr != NULL )
    {
       ...
    }

    return ptr;
}

Answer 9

Asserts are for catching something a developer has done wrong (not just yourself - another developer on your team also). If it's reasonable that a user mistake could create this condition, then it should be an exception.

Likewise think about the consequences. An assert typically shuts down the app. If there is any realistic expectation that the condition could be recovered from, you should probably use an exception.

On the other hand, if the problem can only be due to a programmer error then use an assert, because you want to know about it as soon as possible. An exception might be caught and handled, and you would never find out about it. And yes, you should disable asserts in the release code because there you want the app to recover if there is the slightest chance it might. Even if the state of your program is profoundly broken the user just might be able to save their work.

Answer 10

There was a huge thread regarding the enabling/disabling of assertions in release builds on comp.lang.c++.moderated, which if you have a few weeks you can see how varied the opinions on this are. :)

Contrary to coppro, I believe that if you are not sure that an assertion can be disabled in a release build, then it should not have been an assert. Assertions are to protect against program invariants being broken. In such a case, as far as the client of your code is concerned there will be one of two possible outcomes:

1. Die with some kind of OS type failure, resulting in a call to abort. (Without assert)
2. Die via a direct call to abort. (With assert)

There is no difference to the user, however, it's possible that the assertions add an unnecessary performance cost in the code that is present in the vast majority of runs where the code doesn't fail.

The answer to the question actually depends much more on who the clients of the API will be. If you are writing a library providing an API, then you need some form of mechanism to notify your customers that they have used the API incorrectly. Unless you supply two versions of the library (one with asserts, one without) then assert is very unlikely the appropriate choice.

Personally, however, I'm not sure that I would go with exceptions for this case either. Exceptions are better suited to where a suitable form of recovery can take place. For example, it may be that you're trying to allocate memory. When you catch a 'std::bad_alloc' exception it might be possible to free up memory and try again.

Answer 11

It is not exactly true that "assert fails only in debug mode."

In Object Oriented Software Construction, 2nd Edition by Bertrand Meyer (which is the source to go to for a fully fledged theory on DbC), the author leaves a door open for checking preconditions in release mode. In that case, what happens when an assertion fails is that... an assertion violation exception is raised! In this case, there is no recovery from the situation: something useful could be done though, and it is to automatically generate an error report and, in some cases, to restart the application.

The motivation behind this is that preconditions are typically (though not always) cheaper to test than invariants and postconditions, and that in some cases correctness and "safety" in the release build are more important than speed. i.e. For many applications speed is not an issue, but robustness (the ability of the program to behave in a safe way when its behaviour is not correct, i.e. when a contract is broken) is.

If you're making software for a bank, it might be better to interrupt execution with an alarming message than to transfer $1,000,000 instead of $1,000. But what if you're programming a game? Maybe you need all the speed you can get and if someone gets 1000 points instead of 10 because of a bug that the preconditions didn't catch (because they're not enabled), tough luck. There is no universal answer.

My main point though is that you can have assertions and still get the exceptions automatically if you leave them enabled - at least in Eiffel. I think to do the same in C++ you need to type it yourself.

---

Update: See also When should assertions stay in production code?

Answer 12

I outlined my view on the state of the matter here: http://stackoverflow.com/questions/343605/how-do-you-validate-an-objects-internal-state#343628 . Generally, assert your claims and throw for violation by others. For disabling asserts in release builds, you can do:

• Disable asserts for expensive checks (like checking whether a range is ordered)
• Keep trivial checks enabled (like checking for a null pointer or a boolean value)

Of course, in release builds, failed assertions and uncaught exceptions should be handled another way than in debug builds (where it could just call std::abort). Write a log of the error somewhere (possibly into a file), tell the customer that an internal error occurred. The customer will be able to send you the log-file.

Answer 13

I tried synthesising several of the other answers here with my own views.

Use assertions for cases where you want to disable it in production, erring toward leaving them in. The only real reason to disable in production, but not in development, is to speed up the program. In most cases, this speed up won't be significant, but sometimes code is time critical or the test is computationally expensive. If code is mission critical, then exceptions may be best despite the slow down.

If there is any real chance of recovery, use an exception as assertions aren't designed to be recovered from. For example, code is rarely designed to recover from programming errors, but it is designed to recover from factors such as network failures or locked files. Errors should not be handled as exceptions simply for being outside the control of the programmer. Rather, the predictability of these errors, compared to coding mistakes, makes them more amiable to recovery.

Re argument that it is easier to debug assertions: The stack trace from a properly named exception is as easy to read as an assertion. Good code should only catch specific types of exceptions, so exceptions should not go unnoticed due to being caught. However, I think Java sometimes forces you to catch all exceptions.