Auto run unit test cases in Python

Question

We have a python based web application along with its unit test cases. Our need is to automate the process of running unit test cases. They should run either after every checking OR after every fixed time interval. With minimal effort and time what is best tool that we can use to automate this process. We are using Linux as OS and git as source control.

Answer 1

You're basically looking for continuous integration tools and processes (I mention the term of art because it helps you research the subject in more depth). buildbot is the most popular Python system for the purpose and I would recommend it -- see here for more.

Answer 2

Your goal is to identify which checkins caused tests to fail. Great intuition!

You're using Git, so you may want to start with githooks, which would allow you to create a post-commit script which runs the tests after a commit. If you're feeling gutsy you could even reject users' commits if tests fail — check out this chapter in Pro Git for more info.

Alex is right about continuous integration and Builtbot:

By automatically rebuilding and testing the tree each time something has changed, build problems are pinpointed quickly, before other developers are inconvenienced by the failure. The guilty developer can be identified and harassed without human intervention. By running the builds on a variety of platforms, developers who do not have the facilities to test their changes everywhere before checkin will at least know shortly afterwards whether they have broken the build or not. Warning counts, lint checks, image size, compile time, and other build parameters can be tracked over time, are more visible, and are therefore easier to improve."

Answer 3

Hudson is a good choice for this - I've used it with success in the past. It will monitor the git repository for changes; run the tests and report on failures. It will maintain a history of tests in your project. It has a large number of plugins to support python projects. The Cobertura plugin provides code coverage reports and the violations plugin integrated with pylint to give you an idea of your code quality.

There is a good article on getting it setup on Setting up a Python CI Server on the Rhonabwy.com.