What tools are out there that can perform code coverage analysis at the machine code level rather than the source code level? I'm looking for a possible solution to perform fuzz testing on software that I do not have source code access.
A:
If you have the number of entry points (public methods), you can test the coverage for that. I don't know any tool for that though. Otherwise you would have to test the assembly code coverage, and I don't know if it is possible.
Jader Dias
2010-09-13 13:57:34
+1
A:
I think the IBM Rational test coverage tools instrument object code.
Assuming you had such a tool, but no access to the source, what exactly would code coverage mean, other than 100%?
If you didn't have 100% coverage, you'd know you hadn't exercised something. But you would have no way of knowing what.
Ira Baxter
2010-09-13 19:37:18
I'm interested in using fuzz testing and code coverage analysis to find security vulnerabilities. First, I might run the application under normal operating conditions and identify dusty unused code paths. Then try to identify vulnerabilities under the assumption that the least covered code are likely to have the most vulnerabilities.
Jeremy Powell
2010-09-13 19:50:28
But just exactly what is a "dusty unused path" when you don't have the source? It seems like all you would have is "this instruction wasn't executed" and you'd have to trace out the (downstream branching) path yourself. Even so, you'd have no idea what that path represented; something related to passwords, or something related to printing "disk full"? So what specifically would you do with a list of unexecuted instruction locations?
Ira Baxter
2010-09-13 21:11:55
@Jeremy: (this won't make sense to other SOers): you know Fiona? She knows me. Care to chat by phone?
Ira Baxter
2010-09-13 21:14:27
@Ira That's a good point; I somehow convinced myself that I could divine how to form input to exercise unused code by examining how other input maps into the instructions. Now that I think about it more, it looks quite difficult.
Jeremy Powell
2010-09-14 15:04:56
@Ira: Yes, I know Fiona. I'll connect with you through LinkedIn.
Jeremy Powell
2010-09-14 15:06:03