maven findbugs 'high water mark'

Question

[findbugs is the example here, question is applicable to any such maven plugin]

I attended a build lecture not long ago and a pattern that was talked about that I quite liked was: when adding a new tool to the chain and you start with n violations, you should keep n decreasing (a high water mark) and fail the build only when current check exceeds the last value of n.

findbugs has just been introduced to our build and we were looking for a way to implement this pattern. We couldn't see any way to do it via the plugin configuration, so was curious if anyone out there could mention how they have achieved this. I guess the obvious way is to customize the plugin, but before we go charging ahead, would like to hear thoughts from others.

Answer 1

Findbugs (and all other code metrics plugins I know of) generates an XML file. What I would do is write a maven plugin that specializes in reading these xml files. It would keep a private lookup table somewhere, where it stores per build, per metrics last values.

It would use a common parser interface which you would have to implement for each metric plugin. The config would be something like this:

<plugin>
    <groupId>com.yourcompany</groupId>
            <artifactId>your-plugin-id</artifactId>
            <version>1.0</version>
            <executions>
                    <execution>
                        <id>readmetrics</id>
                        <phase>process-classes</phase>
                        <goals>
                            <goal>analyse-metrics</goal>
                        </goals>
                    </execution>
             </executions>
             <configuration>
                  <metrics>
                      <metric>
                          <type>findbugs</type>
                          <file>${project.reporting.outputDirectory}/findbugs/output.xml</file>
                      </metric>
                      <metric>
                          <type>checkstyle</type>
                          <file>${project.reporting.outputDirectory}/checkstyle/output.xml</file>
                      </metric>
                      <metric>
                          <type>pmd</type>
                          <file>${project.reporting.outputDirectory}/pmd/output.xml</file>
                      </metric>
                  </metrics>
              </configuration>
        </plugin>

Answer 2

The case of FindBugs is in my opinion a bit particular: violations are not just cosmetic, they may be real bugs and should thus be fixed, at least high priority bugs (i.e. when using the High threshold).

Just in case, we follow a similar pattern (our definition of done includes a no technical debt increase) but we didn't implement it in Maven (and we don't fail the build). We use Sonar and its time machine to track metrics evolution (we track the daily evolution). It works well for us, even if it's not as strong as failing the build.

Answer 3

I have raised this issue in the maven findbugs tracker (see http://jira.codehaus.org/browse/MFINDBUGS-115).

Further as part of raising this I have coded a submitted a patch. We are running this patch with success in our large multi-module project.

You could either sync the code and apply the patch by following the instructions on the findbugs-maven-plugin site or hopefully the patch or a derivation of it might be accepted into some future version of the plugin.

Answer 4

Funny that you mention this, because I'm working on exactly this for my employer at the moment. We've spun off an open-source project to hold the work, called dybdob; as it is very much a work in progress, the code that's in the repo right now is rather horrible and very hardcoded/hacky. However, the plan is to do more-or-less exactly what seanizer suggests: parse the XML, keep the counts, and fail if increased.

The first thing I implemented (again: hardcoded, hacky, and undocumented) is a plugin to actually count javac compiler warnings coming out of the build, and break the build if that amount increases. That's now working, and I'm currently working on findbugs, checkstyle, and pmd warnings in parallel.

Would love for you to check it out, and drop me a line if you're interested in helping out (or even just want to see how it evolves).