Code Analysis Tools and Inter-Type-Declarations

Question

I have a maven project generated by Spring Roo and use several tools (checkstyle, pmd etc.) to collect information about my project. (namely I am using codehaus' sonar for this)

Roo makes heavy use of AspectJ Inter Type Declarations (ITD) to seperate concerns like persistence, javabeans-getter/setters etc.

These ITDs are woven in at compile-time so tools like checkstyle and pmd (who work on source-level) have a lot of false positives.

The only solution I currently see, is to deactivate checks for Classes that use ITDs.

Any better ideas?

Knecht

I've googled a while on this one and also spoken to a tool vendor for SCA tools. Seems like this is still a niche problem :-( – Knecht

We use Roo and Sonar too, so I'm interested to see the answers to this question... – Wireman

Ah, feels good not to be the only one. So you know the problems I am talking about? – Knecht

Your problem seems to be that you are asking the SCA tools to process source code for a langauge they don't understand, namely "your target langauge + aspects". Why don't you run the analysis tools over the post aspect-processed code, rather than the original code? Then the type information would be correct, and the language the tools are processing is the one they expect. – Disproportion

Thats is exactly the workaround I'am talking about. The question for me is not if, but how this should be done. For my special case the roo team is already considering the problem so there is hope to me :-) – Knecht

So what's hard about the workaround? Maybe I don't understand the process flow. I thought AspectJ took source code S, applied aspects, and spat out source code S' with woven aspects. Why can't you run the SCA tools trivially over S'? The phrase "woven at compile time" hints that I've got this wrong, and that AspectJ is producing a binary (.class) file directly? – Disproportion

I can't imagine that "deactivating checks" will be very effective. If you are telling your SCA tool that it should ignore or not process certain classes, surely it will have less accurate information about what that class is doing, and that will affects its reasoning about other classes that interact with the one you declared to ignore. So you likely get a chain of reasoning dominoes falling over, and less accurate results not on just the ignored classes, but everywhere else, too. – Disproportion

Good point Ira. That is the reason why I didn't mark Andrew's answer as the correctt one. Its at best a workaround. I am not sure as much as AspectJ is confirmed, but I think there is to different things: Inter Type Declarations and PointCuts but I am not experienced enought with AspectJ to tell the details. – Knecht

We also have a comment on the feasibility from the AspectJ community: old.nabble.com/… – Knecht

Answer 1

This answer would not help you right now, but hopefully it can be of interest for you, as it promises solution for your problem in a near future. I don't know whether you know IntelliJ IDEA - Java IDE from JetBrains, but there is already work being done in this direction, and here is the link to the dedicated issue that you might want to follow: http://youtrack.jetbrains.net/issue/IDEA-26959. Just set a watch on it - and get notified when the feature is implemented. IntelliJ IDEA provides really powerful SCA. So, ITD support should be of a high quality as well.

Abra

Thanks for the hint. Allthough I like to see increasing awareness on the tool-vendor side, I would prefer a solution that can be used from maven2. I want to be able to run my SCA on a headless environment like a CI server. So I cannot afford depending on a specific IDE. – Knecht

It will be possible to run code inspections from maven, and on a CI server, like TeamCity, as it is already done for other analysis features of IntelliJ IDEA. They can be used independently of the IDE, and run remotely, on the server side. – Abra

Answer 2

Doubt it will be a "niche problem" for much longer :-) Hopefully the tool vendors will look at the necessary enhancements.

Equilibrium

Wow. An answer by Rod Johnson. I'm honored :-) I asked some commercial tool vendors, but the reply is still pending. But if I recall right, two of the main Aspect-J Devs also work at springsource, maybe they have an idea for a clever workaround? – Knecht

Answer 3

If you want to reason about the source code after aspects have been woven into the code, you should weave the aspects into the source code rather than the binary code.

Many aspect weavers do binary-code weaving because they don't have access to the information (symbol table, names, types, expression types,...) produced by a compiler front end. So, the hack is, use the virtual machine code produced by the compiler (this stunt basically only works for VM instruction sets like the .net IL and java class codes) which often is easy to decode (nice, regular instruction set) decorated with symbol table information.

But if you can't reason about the binary results of such a weaving process, then you can't be sure than the woven program isn't buggy, which is the point of the OP's original question: "How do I run SCA tools on the (effective) woven source?".

You can fix this two ways:

• Get the community to write SCA tools that process the byte codes rather than the source. This might be hard because the source code may contain information lost in the compilation process.
• A better idea: Get the aspect community to write aspect weavers that operate on source code, and produce source code. This might be hard because getting full language front ends is difficult.

I can't help you make the community make a choice.

I can offer strong encouragement to help the community choose the second way: our DMS Software Reengineering Toolkit. This is a program transformation system that carries out directives of the form of, "if you see this, replace it by that" but honoring the syntax and the semantics of the language by actually applying such changes to compiler data structures produced by full language front ends. (This is the software engineering version of equational substitution in mathematics). The changed data structures can be re-exported as compilable source text, complete with comments.

If you understand what transformations can do in general, you can see that aspect weavers are a special case of program transformation systems. So, it is easy to implement aspect weavers using DMS, and the results are source code, which means you can apply the source-code analysis tools.

I doubt this actually solves the OPs problem of analyzing Roo-generated code in the short term :-{

Disproportion

Answer 4

Both FindBugs and Cobertura do NOT work on the source level, but on the bytecode level. So, you should wave in your aspects statically (that would also improve application start time) at the compile time (e.g. using maven's AspectJ plugin) and not at the load time and then run static analysis tools on the result bytecode.

Barrator

Hi I checked it. And you are right. Most of the false positives I was getting came from pmd and checkstyle which both work on the source level. Nevertheless the problem remains: You cannot analyse code, that you cannot see. – Knecht

Answer 5

Could you add tool-specific annotations/comments to the Java code to suppress the false positives? For example, FindBugs has its own @SuppressWarnings annotation.

Wireman

Hmm interesting. I did not know that. Allthough this will only work on a tool per tool basis, but it might be a starting point. – Knecht

According to the documentation, you are not supposed to touch the AspectJ files. These files are meant to be managed by the shell itself, and may be upgraded when you install newer versions of Roo – Alarm

I meant modifying the Java code, not the ITD; I've updated my answer to be clearer. But as a member of the Roo team, it's gratifying to see that users know not to edit the ITDs! :-) – Wireman