Should Python unittests be in a separate module?

Question

Is there a consensus about the best place to put Python unittests?

Should the unittests be included within the same module as the functionality being tested (executed when the module is run on its own (if __name__ == '__main__', etc.)), or is it better to include the unittests within different modules?

Perhaps a combination of both approaches is best, including module level tests within each module and adding higher level tests which test functionality included in more than one module as separate modules (perhaps in a /test subdirectory?).

I assume that test discovery is more straightforward if the tests are included in separate modules, but there's an additional burden on the developer if he/she has to remember to update the additional test module if the module under test is modified.

I'd be interested to know peoples' thoughts on the best way of organizing unittests.

Answer 1

Personally, I create a tests/ folder in my source directory and try to, more or less, mirror my main source code hierarchy with unit test equivalents (having 1 module = 1 unit test module as a rule of thumb).

Note that I'm using nose and its philosophy is a bit different than unittest's.

Answer 2

I generally keep test code in a separate module, and ship the module/package and tests in a single distribution. If the user installs using setup.py they can run the tests from the test directory to ensure that everything works in their environment, but only the module's code ends up under Lib/site-packages.

Answer 3

if __name__ == '__main__', etc. is great for small tests.

Answer 4

There might be reasons other than testing to use the if __name__ == '__main__' check. Keeping the tests in other modules leaves that option open to you. Also - if you refactor the implementation of a module and your tests are in another module that was not edited - you KNOW the tests have not been changed when you run them against the refactored code.

Answer 5

(1) Where you have to if using a library specifying where unittests should live, (2) in the modules themselves for small projects, or (3) in a tests/ subdirectory in your package for larger projects.

It's a matter of what works best for the project you're creating.

Sometimes the libraries you're using determine where tests should go, as is the case with Django (where you put your tests in models.py, tests.py or a tests/ subdirectory in your apps).

If there are no existing constraints, it's a matter of personal preference. For a small set of modules, it may be more convenient to put the unittests in the files you're creating.

For anything more than a few modules I create the tests separately in a tests/ directory in the package. Having testing code mixed with the implementation adds unnecessary noise for anyone reading the code.

Answer 6

I usually have them in a separate folder called most often test/. Personally I am not using the if __name__ == '__main__' check, because I use nosetests and it handles the test detection by itself.

Answer 7

YES, do use a separate module.

It does not really make sense to use the __main__ trick. Just assume that you have several files in your module, and it does not work anymore, because you don't want to run each source file separately when testing your module.

Also, when installing a module, most of the time you don't want to install the tests. Your end-user does not care about tests, only the developers should care.

No, really. Put your tests in tests/, your doc in doc, and have a Makefile ready around for a make test. Any other approaches are just intermediate solutions, only valid for specific tiny modules.