Checkstyle vs. PMD

Question

We are introducing static analysis tools into the build system for our Java product. We are using Maven2 so Checkstyle and PMD integration come for free. However it looks like there is a large overlap in functionality between these two tools, in terms of enforcing basic style rules.

Is there a benefit from utilizing both of these? I don't want to maintain 2 tools if one will work. If we choose one, which one should we use and why?

We are also planning on using FindBugs. Are there other static analysis tools we should look at?

Update: Consensus seems to be that PMD is preferred over CheckStyle. I don't see a solid reason to use both, and I don't want to maintain 2 sets of rule files, so we will probably aim for PMD exclusively. We'll also be bringing in FindBugs, and perhaps, eventually, Macker to enforce architectural rules.

Retinol

Answer 1

You should definitely use FindBugs. In my experience, the false-positive rate is very low, and even the least-critical warnings it reports are worth addressing to some extent.

As for Checkstyle vs. PMD, I would not use Checkstyle since it is pretty much only concerned with style. In my experience, Checkstyle will report on a ton of things that are completely irrelevant. PMD on the other hand is also able to point out questionable coding practices and its output is generally more relevant and useful.

Valgus

+1 for adding your recommendation of FindBugs. However, I strongly disagree with your opinion on Checkstyle unless you are a lone-wolf developer with your own idiosyncratic style. For teams, agreeing upon a common reasonable subset of rules and then using an automated tool like Checkstyle to automatically enforce them results in code that is readable by all. – Body

While not perfect, FindBugs is the best by far. PMD and checkstyle point you towards downright bad practices. To be avoided at all cost unless you know very well which warnings are valid and which aren't. – Pardoes

Oddly enough I've had the exact opposite experience with PMD vs Checkstyle. PMD often reports false positives if it's something checkstyle or Findbugs didn't find. 7 years can matter a lot though. – Ridenour

Answer 2

Both softwares are useful. Checkstyle will help you during your programming by checking your coding style i.e braces, naming etc. Simple things but very numerous!

PMD will help you by checking more complicate rules like during the design of your classes, or for more special problems like implementing correctly the clone function. Simply, PMD will check your programming style

However, both softwares suffers from similar rules sometimes bad explained. With a bad configuration, you may check things twice or two opposite things i.e "Remove useless constructors" and "Always one constructor".

Baisden

Exactly. IMHO, they're 2 tools aiming to do different things, so I'm not sure if they're even comparable. If you want to enforce a standard coding style among a development team, use Checkstyle. If you want to analyse code for design issues or bad coding practices, use PMD. – Abrahamabrahams

Answer 3

If we choose one, which one should we use and why?

These tools are not competing but are complementary and should be used simultaneously.

The convention type (Checkstyle) is the glue that enables people to work together and to free up their creativity instead of spending time and energy at understanding inconsistent code.

Checkstyle examples:

• Is there javadoc on public methods ?
• Is the project following Sun naming conventions ?
• Is the code written with a consistent format ?

while PMD reminds you bad practices:

• Catching an exception without doing anything
• Having dead code
• Too many complex methods
• Direct use of implementations instead of interfaces
• Implementing the hashcode() method without the not equals(Object object) method

source: http://www.sonarsource.org/what-makes-checkstyle-pmd-findbugs-and-macker-complementary/

Side

I agree that Checkstyle, more focus on code format and forcing developer to follow "Code standard", but it also could detect a lot of bad practices, take a look here, and extension of Checkstyle is more easy for development, but I do agree that it have limitations and will never overcome PMD and FindBug. – Yeh

Answer 4

We use both:

• Checkstyle to make sure that everyone in the team write code in a similar maner
• PMD to find problematic code areas and next refactoring targets

Rodrigorodrigue

Answer 5

If your primary place of use is while developing in eclipse, then CodePro from Instantiations will be best. Earlier it was a commercial tool, but now Google bought Instantiations so CodePro analytix is free now.

Check out http://code.google.com/javadevtools/download-codepro.html

Curlicue

Answer 6

If you reviewed Checkstyle, PMD and Findbugs rule lists, you have seen that all three provide valuable output and all three overlap to a degree and also bring their own, unique rules to the table. This is why tools like Sonar use all three.

That said, Findbugs has the most specific or niche rules (e.g. "Dubious catching of IllegalMonitorStateException" - how often are you likely to run into that?) so it is usable with little or no configuration and its warnings should be taken seriously. With Checkstyle and PMD the rules are more general and style-related so they should only be used with custom configuration files to spare the team from an avalanche of irrelevant feedback ("Tab char on line 5", "Tab char on line 6", "Tab char on line 7"... you get the picture). They also provide powerful tools to write your own advanced rules, e.g. the Checkstyle DescendentToken rule.

When using all three (especially with a tool like Sonar), all of them should be configured separately (takes at least a few days to cover all the rules) while paying attention to prevent duplication (all three tools detect that hashCode() has been overridden and equals() not, for example).

In summary, if you consider static code analysis valuable, rejecting the value any of the three provides makes no sense, but to use all three, you have to invest time to configure them to give you usable feedback.

Halla

Answer 7

Sonar (http://www.sonarsource.org/) is a very useful open platform to manage code quality, and includes Checkstyle, PMD, Findbugs and much more.

This also indicates that all 3 tools have their right to exist...

Acetone

Answer 8

Both tools are configurable and can do just about the same things. That said, if we're talking about out-of-the-box stuff, there is a great deal of overlap, but there are distinct rules/checks as well. For example, Checkstyle has stronger support for checking Javadoc and finding magic numbers, to name a couple. Additionally, Checkstyle has an "import control" feature that looks similar to the functionality of Macker (I've not used Macker).

If there are things that are important to you that Checkstyle does out-of-the-box that PMD doesn't, you might consider a minimal Checkstyle configuration with only those checks. Then institute a policy that the Checkstyle configuration cannot grow, simply remove checks as you implement similar functionality with, say, custom PMD rules.

Also consider that if you decide that the Checkstyle "import control" feature covers what you wanted from Macker, then you could implement PMD/Checkstyle instead of PMD/Macker. Either way it's two tools, but with Checkstyle, you'd get the stuff that PMD doesn't do out-of-the-box "for free."

Seamaid

Answer 9

Checkstyle and PMD both are good at checking coding standards and are easy to extend. But PMD has additional rules to check for cyclomatic complexity,Npath complexity,etc which allows you write healthy code.

Another advantage of using PMD is CPD (Copy/Paste Detector).It finds out code duplication across projects and is not constrained to JAVA.It works for JSP too. Neal Ford has a good presentation on Metrics Driven Agile Development, which talks about many tools that are helpful for Java/Java EE Development

Mcclain

For anyone who reads this...checkstyle now has these checkstyle.sourceforge.net/config_metrics.html checkstyle.sourceforge.net/config_duplicates.html – Meldon

Answer 10

I find Checkstyle and PMD are best for enforcing style issues and simple obvious coding bugs. Although I've found that I like using Eclipse and all the warnings it provides better for that purpose. We enforce stuff by using shared preferences and marking them as actual errors. That way, they never get checked in in the first place.

What I would strongly and enthusiastically recommend is using FindBugs. Because it works at the bytecode level it can check things that are impossible at the source level. While it spits out its fair share of junks, it has found many actual and important bugs in our code.

Slum

Answer 11

And 10 years later ... In 2018 I use all of them Checkstyle, PMD and FindBugs.

Start with FindBugs. Maybe add PMD and Checkstyle later.

Never blindly enforce the default rules !

Steps:

• run one tool with default rules on a project which has a lot of code
• adapt the rules to this project, comment out useless rules with some notes
• focus on the low hanging fruits rules (NPE, logger checks, unclosed resource checks, ...)
• perform some fixes for rules you find worthwhile (one at a time !)
• do this for each tool but not all at once !
• repeat this process

Ideally each project can have separate rules. I like running the rules via the build (via maven plugins) and fail on rule errors once I know a project passes all the rules I defined. This forces developers to take action, because reporting is not enough. From that point on your project is pretty much bullet proof and you could even add more rules later on and/or write custom rules.

Swinney

FYI, SpotBugs is the "spiritual successor of FindBugs", and the SpotBugs documentation is pretty good. As far as I know FindBugs hasn't been updated for years. – Pulsatory

Never heard of SpotBugs, probably because FindBugs + fbcontrib was enough for a long time, good to know there is some replacement – Swinney

There is some discussion about it here: news.ycombinator.com/item?id=12885549 – Swinney

It's also worth noting that tools tend to have configurable sensitivity. Eg when starting out with FindBugs/SpotBugs you might need to choose a High threshold to catch only the most serious bugs, then lower the threshold as you get things fixed. – Ecg

@Ecg yes but even with sensitivity: on a large project too many errors will be found to be fixed in a reasonable amount of time. So what I do instead is add one rule at a time, starting with lowest hanging fruits like potential NP detection, then move on to rules like unclosed resources. – Swinney

Answer 12

One point I have not seen so far is that there are plugins for IDEs that will enforce CheckStyle rulesets on your code, whereas PMD plugins will only report on violations. For example, in a multi-site project over several programming teams, it's important to actively enforce standards, rather than just to report on them.

Both tools have plugins available for IntelliJ, NetBeans, and Eclipse (in my view this covers most usage). I'm not as familiar with NetBeans, so can only comment on IntelliJ and Eclipse.

Anyway, the PMD plugins for IntelliJ, and Eclipse, will generate reports on demand on PMD violations within the project codebase.

The CheckStyle plugins, on the other hand, will highlight violations on the fly, and can (at least for IntelliJ, I have less experience with Eclipse) be configured to automatically convert some issues (e.g. for 'OneStatementPerLine', will place CR-LF between statements, for 'NeedBraces', will add braces where they are missing, etc.). Obviously, only the simpler violations can be automatically fixed, but it's still a help on legacy projects, or projects located over several locations.

'On demand' for PMD means that the developer must consciously decide to run the report. Whereas Checkstyle violations are automatically reported to them as they develop. While PMD does contain a more extensive ruleset, in my mind the automatic enforecement/reporting of violations in IDEs is worth the hassle of maintaining 2 sets of rules.

So for any projects I work on, we use both tools, Checkstyle enforced in the IDE, PMD reported in the IDE, and both reported and measured in builds (through Jenkins).

Hakluyt

There are also ways to integrate it in the build and to make it fail on violation (with maven for example). I have done this for Checkstyle, PMD and FindBugs. As you said reporting is not enough. – Swinney

Answer 13

Take a look at qulice-maven-plugin that combines together Checkstyle, PMD, FindBugs and a few other static analyzers, and pre-configures them. The beauty of this combination is that you don't need to configure them individually in every project:

<plugin>
  <groupId>com.qulice</groupId>
  <artifactId>qulice-maven-plugin</artifactId>
  <version>0.15</version>
  <executions>
    <execution>
      <goals>
        <goal>check</goal>
      </goals>
    </execution>
  </executions>
</plugin>

Sealey

How do I configure it to get a report (in some usable format)? Now it only spits it to the console even if I configure log4j. I see that there is a bug report which might be related, but I'm not sure. – Bungle

We are thinking the same but in order to fix it I need it to be highlighted in my code or something similar. At least your settings.jar helped. – Bungle

Answer 14

I would echo the comment that PMD is the more current product for Java style/convention checking. With respect to FindBugs, many commercial development groups are using Coverity.

Lewellen

Answer 15

PMD is what I find more people referring to. Checkstyle was what people were referring to 4 years ago but I believe PMD is maintained more continuously and what other IDEs/plugins choose to work with.

Revoke

True in 2008, but today Checkstyle has picked up a lot of speed. – Guitar

Answer 16

I have just started to use Checkstyle and PMD. To me, PMD is more easier to create customized rules for things such that whether there exists System.gc(), Runtime.gc(), as long as you can write the XPath Query which is also not difficult at all. However, PMD has not shown me that it has the feature to show column number. So for things like check column limits. You might would like to use Checkstyle.

Behl

Answer 17

PMD is the finest tool when compare with checkstyles. Checkstyles may not have the capability to analyse the code while PMD offering many features to do so! Offcourse PMD has not released rules for javadoc, comments, indentations and etc. And by the way i am planning to implement these rules.......thanx

Australopithecus

One good thing about checkstyle is it allows some flexible rule like RegexpSingleline... – Affecting