Python - doctest vs. unittest

Question

I'm trying to get started with unit testing in Python and I was wondering if someone could inform me of the advantages and disadvantages of doctest and unittest. What conditions would you use each for?

Answer 1

I use unittest almost exclusively.

Once in a while, I'll put some stuff in a docstring that's usable by doctest.

95% of the test cases are unittest.

Why? I like keeping docstrings somewhat shorter and more to the point. Sometimes test cases help clarify a docstring. Most of the time, the application's test cases are too long for a docstring.

Answer 2

If you're just getting started with the idea of unit testing, I would start with doctest because it is so simple to use. It also naturally provides some level of documentation. And for more comprehensive testing with doctest, you can place tests in an external file so it doesn't clutter up your documentation.

I would suggest unittest if you're coming from a background of having used JUnit or something similar, where you want to be able to write unit tests in generally the same way as you have been elsewhere.

Answer 3

I almost never use doctests. I want my code to be self documenting, and the docstrings provide the documentation to the user. IMO adding hundreds of lines of tests to a module makes the docstrings far less readable. I also find unit tests easier to modify when needed.

Answer 4

I use unittest exclusively; I think doctest clutters up the main module too much. This probably has to do with writing thorough tests.

Answer 5

Using both is a valid and rather simple option. The doctest module provides the DoctTestSuite and DocFileSuite methods which create a unittest-compatible testsuite from a module or file, respectively.

So I use both and typically use doctest for simple tests with functions that require little or no setup (simple types for arguments). I actually think a few doctest tests help document the function, rather than detract from it.

But for more complicated cases, and for a more comprehensive set of test cases, I use unittest which provides more control and flexibility.

Answer 6

I prefer the discovery based systems ("nose" and "py.test", using the former currently).

doctest is nice when the test is also good as a documentation, otherwise they tend to clutter the code too much.

Answer 7

Both are valuable. I use both doctest and a further unit testing framework (nose) taking the place of unittest. I use doctest for cases where the test is giving an example of usage that is actually useful as documentation. Generally I don't make these tests comprehensive, aiming solely for informative. I'm effectively using doctest in reverse: not to test my code is correct based on my doctest, but to check that my documentation is correct based on the code.

The reason is that I find comprehensive doctests will clutter your documentation far too much, so you will either end up with either unusable docstrings, or incomplete testing.

For actually testing the code, the goal is to thoroughly test every case, rather than illustrate what is does by example, which is a different goal which I think is better met by other frameworks.

Answer 8

Another advantage of doctesting is that you get to make sure your code does what your documentation says it does. After a while, software changes can make your documentation and code do different things. :-)