What is your favourite Code Coverage tool(s) (Free/non-free) and how do you use them effectively?
There are several options available, such as:
I have CodeCover telling me various chunks of my code are 58% covered etc. But how does this help me write better code?
Code coverage is not neccesarily about writing better code but writing better applications. Covering more of your code with unit tests should in theory lead to more robust systems, more defects found earlier in the dev life cycle, less accidental breaking changes etc.
A high level of code coverage can also give you a high degree of confidence when making potentially 'dangerous' changes.
http://cobertura.sourceforge.net/ in eclipse is very nice in my oppinion. I've also used EclEmma alot and like that very much as well. Actually EclEmma would be my first choice
It doesnt really help you write better code per se. You just get an objective number. Maybe listen to the Jeff Atwood on Herding Code podcast regarding unittesting and coverage in Stack Overflow
It helps you to write better code by illustrating which parts of your code are exercised by your tests. Basically, it tells you how comprehensive your tests are. The assumption is that automated tests increase the quality of your code (which I believe they do, over time).
Note: You can't just rely on code coverage. If you aren't also looking at branch coverage, you don't really have the whole picture.
Cobertura gives quite a lot of statistics that you may or may not know how to interpret. My favorite tool so far is EclEmma - which is the Eclipse plugin for Emma. I found that it was the least overhead for setup and usage. What I like the most about it is you can run your jUnits and it will color-code which lines it covered as well as showing the percentages per class/method etc.
It's a commercial tool, but I'll thrown in a recommendation of Clover from Atlassian.
Others have done a good job of answering the "how to use it" questions.
The one built into IntelliJ IDEA (based on EMMA, I think). The integration with the IDE is remarkable.
I use Cobertura. I prefer it to Emma because it does branch coverage as well as line coverage and because I think the HTML reports it generates are nicer.
An example of why branch coverage is important is code that uses the ternary operator (a ? b : c). If the tool does only line coverage, this statement will be considered 100% covered even if it always executes the same branch. With branch coverage it will be only 50% covered unless both branches have been executed.
Code coverage doesn't directly help you write better code, but it does help you write better tests, which in turn help you write better code. Code coverage will uncover execution paths that are not being exercised by your tests. As well as indicating where you can improve your tests, this can also identify pieces of code that can be removed (since they are not being used).
If you use a continuous integration tool, such as Hudson, you can plot your coverage scores over time, which can be useful for indentifying trends.
I've just started using Python and I've found coverage.py to be very useful, especially when coupled with unit testing.
I try to keep coverage at 100%. Now, that's difficult to explain here, but I've recently blogged about it here.
For me, its about making me think about why, when I'm using TDD, lines are not covered when I'm only supposed to write enough code to get the next test to pass.
If you want me to vote on your suggestions, then eclEmma. It just works!
And yes, I know this is a python tool when most of the discussion seems to be around java. Sorry.
I haven't used it much, but I like the concept of Crap4j. It combines code coverage and complexity measurements to let you check that you've covered all your risky code. If you've got a bunch of getters and setters or other simple code, it's probably less important to test that than some complex business rules in another class. This is particularly useful if you're trying to add test coverage to existing code.
Code coverage does not help you write better code.
It helps you write better tests.
Better tests help you write better code.
I would also go so far as to add that Code Coverage, even 100% coverage only tells you that you HAVE executed some code. It doesn't say that your intent was correct or that what you intended to have happen actually happen.
Sorry if that's a snarky comment, but I think it's a good reminder.
Disclaimer: I'm an Atlassian
Clover combines coverage and complexity via the "Project Risks" tag cloud. It displays uncovered classes redder, and complex classes bigger to make spotting potential project risks easier. This provides some indication of which classes to test next.
Also, "per-test coverage" allows you to view the tests that covered each individual line of code. Click on the dark green bars in the margin of this sample report:
Clover is free for open source projects, and free for a 30 day evaluation.
I represent Semantic Designs, so don't view this as "my favorite tool". You be the judge.
See SD Test Coverage Tool. It handles many languages, including Java. A GUI display shows your code with overlayed executed/non executed indications. If you have multiple conditionals in a line (if, for, while, ...) you will see coverage data for them all seperately; most test coverage tools will only tell you about whether that line has been executed. That same display produces summary results at the method/class/file, etc. levels. It has extremely low probe overhead, and it will handle extremely large systems as monoliths or in (even overlapping) segments. You can compose many seperate test run results to get a single result, compare test runs, etc.
Check this CodePro AnalytiX out, haven't tested it deeply, but so far seems really handy, easy to use with bunch of nice features.