condition coverage in python

Question

Is there any tool/library that calculate percent of "condition/decision coverage" of python code. I found only coverage.py but it calculates only percent of "statement coverage".

Answer 1

Are you looking for cyclomatic complexity (Wikipedia)? It basically computes the number of paths through a piece of code. There are some projects to compute that for Python code, for example PyMetrics or this one. Google will certainly bring up more.

But I don't know of any further integration with unit tests that will show you the coverage.

Answer 2

I don't know of any branch coverage tools for Python, though I've contemplated writing one. My thought was to start with the AST and insert additional instrumentation for each branch point. It's doable, but there are some tricky cases.

For example,

raise SomeException(x)

Branch coverage for this needs to check that SomeException(x) was fully instantiated and didn't raise its own exception.

assert x, "Oh No!: %r" % (x, y)

This needs to check that the text on the right side of the assertion statement is fully evaluated.

return args.name or os.getenv("NAME") or die("no name present")

Each of the first two terms has to be checked for the true/false path, but not the last. In fact, the last might not even return.

There were a lot of cases to worry about and I had no pressing need for it other than curiosity so I didn't go anywhere with it. I was also wondering if I would get a lot of false positives where I would need some way to repress specific warnings.

If you want to try this route, start with Python 2.6 or 3.0. In those releases the AST module is documented and you can create your own AST nodes before generating the code or .pyc file.

Answer 3

The very same maintainer of coverage.py has an article discussing a way to get coverage information at the bytecode level. The method is a bit kludgey: it involves re-assembling .pyc files with tweaked line numbers. However, it provides about as much granularity in coverage measurement as you could ask for.

Answer 4

I haven't used it myself, but if you are willing to replace coverage analysis with mutation testing, I've heard of a mutation tester called "pester".

While I was doing googling, I also came across a list of python testing tools which mentions some possible code coverage tools.

Answer 5

Parse and modify the AST is the right answer, IMHO. See this paper for a complete description of what you need to do: "Branch Coverage Made Easy for Arbitrary Languages"

http://www.semanticdesigns.com/Company/Publications/TestCoverage.pdf

Answer 6

Coverage.py now includes branch coverage.

For the curious: the code is not modified before running. The trace function tracks which lines follow which in the execution, and compare that information with static analysis of the compiled byte code to find path possibilities not executed.