How to implement a code coverage tool using Win32 Debugging API

Question

I am trying to understand how to implement a Code Coverage tool using the Win32 Debugging API.

My thinking has been to utilize the Win32 Debugging API to launch a process in debug mode - and track what CPU instructions has been executed. After having tracked all CPU instructions I would then use the map file to map it to what source code lines were executed.

As far as I understand, there would be two ways of knowing what CPU instructions have been executing.

1. Would be to launch the process in debug mode - set all threads in single step mode and let the debugging app note all instructions that has been executed

2. Would be make a more intelligent approach where you would know a lot more about x86 instructions and basically replace the next branch instruction with a breakpoint. Then keeping track of the delta instructions between the two breakpoints.

Update - new suggested approaches inspired by Michael's response:

1. Start with the map file and insert breakpoints for the beginning of each line and let the debug framework be notified every time a breakpoint hits.

2. Start with the map file - binary instrumentation to insert a "hook" that get called at entry of each source line - avoiding the callback through the debugger framework.

3. Using a VM Technology - such as VMware to find out what instructions in a particular process was executed - I don't fully understand this approach...

Could someone validate one of the approaches above or maybe suggest an alternative - please note that the use case is line-by-line code coverage and not performance profiling - thus we need to know if each single source line is visited.

My primary goal (although no particular plan is in place...) would be to create a simple code coverage tool for Delphi primarily.

Thanks!

Answer 1

One approach is hooking all api calls and fucntion calls to compare with table made from the source. Thus you discovers what is covered.
There is many api for hooking, one is Trappola API hooking

Answer 2

This could work - each single step event will create an exception and you could record the hit IP address in your map of executed code lines.

Unfortunately, I imagine this would be glacially slow. It'd be incredibly inefficient, as each single line of code results in 1000's of times more work, as an exception is generated, trapped, a message sent to your debugger, and then a round trip back after you record the hit. It might be better to try to set breakpoints instead for each covered line and clear them after they are hit. That'd be faster, but most likely still very slow.

The core problem is you're trying to use the debugger as a code coverage tool which it is not intended for. A quick search shows several code coverage tools for Delphi on the Internet.