How to organize python test in a way that I can run all tests in a single command?

Question

Currently my code is organized in the following tree structure:

src/
    module1.py
    module2.py
    test_module1.py
    test_module2.py
    subpackage1/
        __init__.py
        moduleA.py
        moduleB.py
        test_moduleA.py
        test_moduleB.py

Where the module*.py files contains the source code and the test_module*.py contains the TestCases for the relevant module.

With the following comands I can run the tests contained in a single file, for example:

$ cd src
$ nosetests test_filesystem.py
..................
----------------------------------------------------------------------
Ran 18 tests in 0.390s

OK

How can I run all tests? I tried with nosetests -m 'test_.*' but it doesn't work.

$cd src
$ nosetests -m 'test_.*'

----------------------------------------------------------------------
Ran 0 tests in 0.000s

OK

Thanks

Answer 1

This is probably a hotly-contested topic, but I would suggest that you separate your tests out from your modules. Set up something like this...

Use setup.py to install these into the system path (or you may be able to modify environment variables to avoid the need for an "install" step).

foo/
    module1.py
    module2.py
    subpackage1/
        __init__.py
        moduleA.py
        moduleB.py

Now any python script anywhere can access those modules, instead of depending on finding them in the local directory. Put your tests all off to the side like this:

tests/
    test_module1.py
    test_module2.py
    test_subpackage1_moduleA,py
    test_subpackage2_moduleB.py

I'm not sure about your nosetests command, but now that your tests are all in the same directory, it becomes much easier to write a wrapper script that simply imports all of the other tests in the same directory. Or if that's not possible, you can at least get away with a simple bash loop that gets your test files one by one:

#!/bin/bash
cd tests/
for TEST_SCRIPT in test_*.py ; do
    nosetests -m $TEST_SCRIPT
done

Answer 2

I'll give a Testoob answer.

Running tests in a single file is like Nose:

testoob test_foo.py

To run tests in many files you can create suites with the Testoob collectors (in each subpackage)

# src/subpackage?/__init__.py
def suite():
  import testoob
  return testoob.collecting.collect_from_files("test_*.py")

and

# src/alltests.py
test_modules = [
    'subpackage1.suite',
    'subpackage2.suite',
]

def suite():
    import unittest
    return unittest.TestLoader().loadTestsFromNames(test_modules)

if __name__ == "__main__":
    import testoob
    testoob.main(defaultTest="suite")

I haven't tried your specific scenario.

Answer 3

Whether you seperate or mix tests and modules is probably a matter of taste, although I would strongly advocate for keeping them apart (setup reasons, code stats etc).

When you're using nosetests, make sure that all directories with tests are real packages:

src/
    module1.py
    module2.py
    subpackage1/
        __init__.py
        moduleA.py
        moduleB.py
tests/
    __init__.py
    test_module1.py
    test_module2.py
    subpackage1/
        __init__.py
        test_moduleA.py
        test_moduleB.py

This way, you can just run nosetests in the toplevel directory and all tests will be found. You need to make sure that src/ is on the PYTHONPATH, however, otherwise all the tests will fail due to missing imports.

Answer 4

I don't know about nosetests, but you can achieve that with the standard unittest module. You just need to create a test_all.py file under your root directory, then import all your test modules. In your case:

import unittest
import test_module1
import test_module2
import subpackage1
if __name__ == "__main__":
    allsuites = unittest.TestSuite([test_module1.suite(), \
                                test_module2.suite(), \
                                subpackage1.test_moduleA.suite(), \
                                subpackage1.test_moduleB.suite()])

each module should provide the following function (example with a module with two unit tests: Class1 and Class2):

def suite():
    """ This defines all the tests of a module"""
    suite = unittest.TestSuite()
    suite.addTest(unittest.makeSuite(Class1))
    suite.addTest(unittest.makeSuite(Class2))
    return suite
if __name__ == '__main__':
   unittest.TextTestRunner(verbosity=2).run(suite())

Answer 5

If they all begin with test then just nosetest should work. Nose automatically searches for any files beginning with 'test'.