How to get started "writing" a code coverage tool?

Question

Looking for books or other references that discuss actually "how" to write a code coverage tool in Java; some of the various techniques or tricks - source vs. byte code instrumentation. This is for a scripting language that generates java byte code under the hood.

Answer 1

Thxm, Mc! http://asm.objectweb.org/ is another one. Excellent documentation on byte code instrumentation, but nothing "directly" aimed at writing a coverage tool - just some hints or ideas.

Answer 2

Is your scripting language bytecode generating? Does it generate debug metadata? If so, bytecode instrumentation is probably the way to go. In fact existing tools like will probably work; perhaps with minimal modification (the typical problem is the tools are written to work with Java and assume com.foo.Bar.class corresponds to com/foo/Bar.java. Unwinding that assumption can be tedious.) EMMA is a ClassLoader that does byte-code re-writing for code-coverage collection in Java. The coding style is a little funky, but I recommend reading the source for some ideas.

If your scripting language is interpreted then you will need something higher-level (source level) that hooks into the interpreter.

Answer 3

You can also get the source from a Open Source code coverage tool and learn from it.

Answer 4

You might also want to use something like BCEL to analyse which lines of source actually exist in the byte-code. You don't want to report that things like blank lines and comments haven't been covered.

Answer 5

If you're talking about ColdFusion (which I assume you are from the tags) then I'm not sure this is doable but I may be very wrong here...

IIRC, When CF compiles it essentially compiles into a interpreted form of the CFML as a plain old java source file, this is then compiled into the class. Therefore, any instrumentation that you may have will apply to the intermediary version rather than the CFML itself.

Saying that though, Adobe have got the CF debugger now which can step though code, so please prove me wrong - I'd love code coverage in CFML.

Answer 6

See the paper "Branch Coverage for Arbitrary Languages Made Easy" at http://www.semanticdesigns.com/Company/Publications/TestCoverage.pdf