Anyone using VeraCode for static analysis?

Question

My company is looking at using VeraCode for some independent static analysis of our binaries. Anyone out there using them?

I would also be interested to hear from anyone using their dynamic analysis services as well.

Answer 1

I didn't know VeraCode yet, thanks for the hint. I usually prefer static-analysis-tools, that I can execute on my own hardware, like PMD, Checkstyle and Findbugs.

Answer 2

Mnementh brings up the issue I am concerned with.

I personally believe that static analysis is most effective when the developers themselves can execute the tool against their code. Then they can see the problems being reported very soon after they make them. Otherwise it becomes a find and fix kind of issue.

But we will see what they come up with. It can't hurt to have another analysis of the code to see what is potentially an issue.

Answer 3

We had Veracode in for an evaluation and they were very impressive and are high on my list for providing security analysis for our app.

Their model has lots of advantages - they don't need source (so they can test 3rd party libs), can simulate code execution, can centralize their experience with all the apps they test to help eliminate false positives, but the biggest advantage is that we don't need to hire a penetration test expert. The only reason I might consider going another way (buying a tool) is that if we wind up doing lots of scans, Veracode might become too expensive.

There's also something to be said for having the security testers know the app well, so if I had a trained security testing staff, I'd be more likely to buy a tool rather than use Veracode (which uses a SAAS model). Since I don't have that in-house expertise, Veracode appears to be a good "low cost of entry" way to do security testing.

Answer 4

I must admit considerable puzzlement at doing static analysis of object code vs. source code.

The main problem with any static analysis is that that results must be conservative. If you can't tell what is happening, you have to assume the worst. (Well, you can assume the best, but then why do you need a tool?) Machine code is notoriously hard to analyze. If I have a jmp indirect via a register (or any of them any indirect accesses possible with index registers), it is likely pretty hard to predict where it will go. (CPU designers kill themselves to get 90% effective branch predication rates). If I don't know where the program goes, how can I decide what it does? If I don't know what it does, I have to predict conservatively that it is dangerous. That's a cheap prediction (and in fact you don't need a tool to do it), but false positives can waste a huge amount of time.

So, what does Veracode produce as diagnostics? How good are they? What's the false positive rate?

It seems to me that that source code analysis is far superior. Source code doesn't contain jump indirect doesn't cause immediate conservative analyses. Array accesses are assumed (with a necessary check) to access the array, not just any location in memory. It seems that if I have the source code, and I have the object code, I should be able to make much more precise analyses of the source code than the object code. (full disclosure, I'm biased, check my bio, but my bias comes from this very argument)

So the only excuse for object code analysis is that you don't own the source. In this case, you should ask your vendor to analyze his code rather than you analyzing his object code. It smacks of looking under the lamppost.

Maybe Veracode is spectacular. Maybe not. I'd have to hear very strong arguments as to why they can be superior to source code analysis, and I admit I don't see how there can be any.