Run all unit test in Python directory

Question

So I have a directory that contains my Python unit test. Each unit test module is of the form "test_*.py". I am attempting to make a file called "all_test.py" that will, you guessed it, run all files in aforementioned test form and return the result. I have tried two methods so far, both have failed, I will show the two methods, and hope someone out there knows how to actually do this correctly. Thank you.

For my first valiant attempt, I thought "If I just import all my testing modules in the file, and then call this unittest.main() doodad, it will work, right?" Well, turns out wrong.

import glob
import unittest

testSuite = unittest.TestSuite()
test_file_strings = glob.glob('test_*.py')
module_strings = [str[0:len(str)-3] for str in test_file_strings]

if __name__ == "__main__":
     unittest.main()

This did not work, the return result I got was.

$ python all_test.py 

----------------------------------------------------------------------
Ran 0 tests in 0.000s

OK

For my second try, I though, ok, maybe I will try to do this whole testing thing in a more "manual" fashion. So I attempted to do that below.

import glob
import unittest

testSuite = unittest.TestSuite()
test_file_strings = glob.glob('test_*.py')
module_strings = [str[0:len(str)-3] for str in test_file_strings]
[__import__(str) for str in module_strings]
suites = [unittest.TestLoader().loadTestsFromName(str) for str in module_strings]
[testSuite.addTest(suite) for suite in suites]
print testSuite 

result = unittest.TestResult()
testSuite.run(result)
print result

#Ok, at this point, I have a result, how do I display it as the normal unit test
#command line output?
if __name__ == "__main__":
    unittest.main()

This also did not work, but it seems so close!

$ python all_test.py 
<unittest.TestSuite tests=[<unittest.TestSuite tests=[<unittest.TestSuite tests=[<test_main.TestMain testMethod=test_respondes_to_get>]>]>]>
<unittest.TestResult run=1 errors=0 failures=0>

----------------------------------------------------------------------
Ran 0 tests in 0.000s

OK

Seems so close! I seem to have a suit of some sort, and I can execute the result. I am a little concerned about the fact that it says I have only "run=1", seems like that should be "run=2", but it is progress. But how do I pass and display the result to main? Or how do I basically get it working so I can just run this file, and in doing so, run all the unit test in this directory?

Answer 1

You could use a test runner that would do this for you. nose is very good for example. When run, it will find tests in the current tree and run them.

Updated:

Here's some code from my pre-nose days. You probably don't want the explicit list of module names, but maybe the rest will be useful to you.

testmodules = [
    'cogapp.test_makefiles',
    'cogapp.test_whiteutils',
    'cogapp.test_cogapp',
    ]

suite = unittest.TestSuite()

for t in testmodules:
    try:
        # If the module defines a suite() function, call it to get the suite.
        mod = __import__(t, globals(), locals(), ['suite'])
        suitefn = getattr(mod, 'suite')
        suite.addTest(suitefn())
    except (ImportError, AttributeError):
        # else, just load all the test cases from the module.
        suite.addTest(unittest.defaultTestLoader.loadTestsFromName(t))

unittest.TextTestRunner().run(suite)

Answer 2

Well by studying the code above a bit (specifically using TextTestRunner and defaultTestLoader), I was able to get pretty close. Eventually I fixed my code by also just passing all test suites to a single suites constructor, rather than adding them "manually", which fixed my other problems. So here is my solution.

import glob
import unittest

test_file_strings = glob.glob('test_*.py')
module_strings = [str[0:len(str)-3] for str in test_file_strings]
suites = [unittest.defaultTestLoader.loadTestsFromName(str) for str
          in module_strings]
testSuite = unittest.TestSuite(suites)
text_runner = unittest.TextTestRunner().run(testSuite)

Thanks for everyones help. Yeah, it is probably easier to just use nose than to do this, but that is besides the point. :)