What is stopping you from using static analysis tools?

Answer 1

Lack of support from the developers, believe it or not. Everyone believes either that their case is a special case and should be exempted, or that worrying about the output of those tools is the programming equivalent of picking nits. The sad part is, most of those nits will grow into real bugs!

Answer 2

I can't speak for the business world, but one of the reasons there's been a slow uptake of static analysis in open-source programs is cost and licensing. The best tools are expensive and have very restrictive licenses, so it's hard for an open-source project to use them. This is getting better - Coverity offers to scan open-source projects for free, for example, and the KDE project developed their own Krazy software to do daily checks.

Answer 3

• I regularly use FindBugs, since it seldom delivers false positives.
• Most other tools show hundreds of warnings, many of questionable utility
• Many tools don't have the ability to disable checks selectively (e.g. for unit tests or legacy code that cannot be changed)
• A style checking tool must be used from the outset of a project, otherwise it creates a mountain of warnings that lead to developer frustration.

Answer 4

What Branan said applies doubly to the business world. Both Coverity and Klocwork are expensive. Add to that what rhinesj said about resistance from developers, and you have a tough internal sell.

Having looked at Coverity and Klocwork, I think they're worth the cost. Yes, there are some false positives, etc. But the kinds of bug they catch are often those that are really weird corner cases, which your QA guys have difficulty reproducing, but your customers find sooooo easily when they decide to upgrade. Late on Friday night. On December 24th. And they call you...

Answer 5

The cost of using this tools on large amount of existing code is very expensive, and unless there is marketing demand for this we can't afford 10%-20% of already invested time for each module. However we have a customer that run lint on his code and after integrating our code on our code as well. The noise was very small and only few minor fixes was done after filtering out false positive warnings that was done by customer.

What we do is 2 stage code review :

1. Every line committed to cvs reviewed by another team member before actual commit
2. Offline review usually done by team leader/component owner.

So once we will start new product i will definitely use static code analyzer. At least it will reduce number of thing we need to check in code review process :). But 2 stages review is very effective. We did statistic in the past and clearly see that time invested in review return the investment very fast.

BTW i write C code so i think the question is related to every language.I missed the Java part at first :)

Answer 6

In my organization we don't use FindBug mostly because we work with other people code. And our changes are mostly minor. We care about bugs in our code not in those made by our competitors.

So the answer is legacy code.

OTOH: Projects that are developed by us from scratch are checked by FindBug.

I don't use CheckStyle mostly because it emits loads of useless warnings like: 'no space after parenthesis'. My code looks good and it's reasonably easy to read.

And I never got to configuring checkstyle.

So the answer is no time to configure.

Answer 7

For me it was all unfamiliarity. I had heard FindBugs mentioned before, but never knew what it was until reading this question. I had assumed (incorrectly) that like many other static analysis tools it would cost thousands of dollars.

Now I'm using it!

Answer 8

Believe it or not: it is ignorance and arrogance. Ignorance comes from the management side and the later from the developers (some people don't like the unconvenient truth that there is a bug in their code somewhere).

There is no such thing as a silver bullet, and such tools are definitely the good example. It may happen you get too many false-positives in PMD or FindBugs (but this can be fixed by tweaking the rules). Checkstyle might produce too many errors / warnings on formatting issues (but this is exactly the point, whole team should use the same formatting rules in the IDE - anyone not using them is asking for SCM merging hell). But it is better to be prepared. One day a major refactoring will be needed and then you can see what such tools are good for.

PMD, FindBugs, Checkstyle, Clover / Cobertura and others might be sometimes a real pain to work with (imagine the situation when you are racing for time in bugfixing but you also need to put the coverage up to the required level), but are extremely useful in creating a safety net for developers themselves. NOT using them, especialy in a big team with high junior dev ratio is a suicide.

Answer 9

Back in college I used ESC/Java for static analysis, whenever I encounter problem with it, the only place I can look for answer was ESC/Java's manual; couldn't find any useful discussion from Google. Setting up ESC/Java was also not easy, had to deal with a lot of path problem. So after using ESC/Java I thought static analysis is impractical.

But everything changed after I used FindBug. I was impressed with the fact that it has a GUI and ease of use.

I guess what I am getting at is that the reason that would keep me from using a tool is if it lacks popularity (thus, lack of support from other peers) and lacks usability.

Answer 10

1. Inability to apply analysis tools to a legacy code without rewriting everything. Either you start with some standards or you are out of the tools at all.
2. a limited understanding of such tools.
3. conflict of interests between programmers - too many coding standards.
4. lack of real value of such tools - in most cases they do only shallow scan. Indentation and naming conventions are of limited value in large projects, as opposed to deep dependency control which have not been understood completely yet.

Most programmers perceive such tools as limiting their creativity. They see their personal goals and not the project goals. They like tools, but only those, which they have found themselves.

A story from some big project:

Some people had decided to use some static analysis tools. Nobody understood why, but it was enforced. Then came the time for an intermediate milestone delivery. So people started hacking in order to get the demo shining. People convinced management that for the demo the analysis tools are not that much important. So they left them for a while. After the milestone delivery they started regular development based on the shiny demo version. They were never able to get back to their static analysis tools.

Link to our research (not related to the story): www.mimuw.edu.pl/~chrzaszc/papers/JC-TS-AGS-AS_Minik-A-tool-for-maintaining-proper-Java-code-structure.pdf

Answer 11

Another aspect to consider is whether you are using another style or standard that conflicts in some way with the tool. While these tools definitely have value, they can become a burden in these cases.

For example, if doing Java GUI development using Netbeans, you'll find that a lot of the auto-generated code may cause a warning in PMD for example (I don't have examples off hand).

If that's the case, and your choice is to either go with the way the IDE does things or the static analysis tool, if you chose the latter it will be a massive pain, for not much gain, as these types of warnings are usually either wrong or worthless.

You can then take time to configure the tools to ignore certain warnings, but there is inedequate configuration options, e.g. you can usually only either use a rule or not, but not define rules for when they should apply.

So to sum things up, while static analysis tools have value, in some cases the drawbacks may outweigh the benefits. This could change if the tools become more customizable, and more intelligent with their rules.