Cascade In-Memory Compilation with javax.tool

Question

Eclipse's JDT compiler provide an interface INameEnvironment which defines method findType(...) enable you to do cascade compilation. Curiously I would like to know if there are any means to do it using standard JDK compiler toolkit?

Note, the scenario is a template engine which do in memory compilation for template file generated classes which have inter-dependencies, and it cannot forecast the order you encountered a template file, thus Foo might needs to be compiled first before it's parent Bar compiled already, therefore you need a mechanism to do cascade compilation, meaning during compilation of Foo you need to generate another source Bar and compile it first in order to continue Foo's compilation: some code like the follows:

private NameEnvironmentAnswer findType(final String name) {
    try {
        if (!name.contains(TemplateClass.CN_SUFFIX)) {
            return findStandType(name);
        }

        char[] fileName = name.toCharArray();
        TemplateClass templateClass = classCache.getByClassName(name);

        // TemplateClass exists
        if (templateClass != null) {

            if (templateClass.javaByteCode != null) {
                ClassFileReader classFileReader = new ClassFileReader(templateClass.javaByteCode, fileName, true);
                return new NameEnvironmentAnswer(classFileReader, null);
            }
            // Cascade compilation
            ICompilationUnit compilationUnit = new CompilationUnit(name);
            return new NameEnvironmentAnswer(compilationUnit, null);
        }

        // So it's a standard class
        return findStandType(name);
    } catch (ClassFormatException e) {
        // Something very very bad
        throw new RuntimeException(e);
    }
}

Aholla

Answer 1

Based on our comment conversation, I think the answer is a clear: no, you can't do that with the JDK compiler. It does give you a hook when it requests the package, but not the specific class dependency.

About as close as you can get so far as I know is:

Here's a nice article with code though it needs to be adapted to handle in memory classes. Specifically the issue you're describing is handled by the JavaFileManager.list(...) method. You have to return back JavaFileObjects here that you have cached in memory. You'll most likely need to create a subclass of ForwardingJavaFileManager as described in the article--though modified to handle the cached classes you are working with.

You can use that to compile something. If it returns errors, use regex to find out what is missing. After generating/compiling the code for the missing thing, retry compiling the original code.

NOTE: It does ask for the FQN of the dependent class as the packageName argument in ForwardingFileManager.list(...) at some point. I haven't tried to return the class at that point. It might not work because the package would mismatch, but maybe it would.

Baron

The point of JDT's INameEnvironment.findType() is it allows me to do cascade compilation, say, I have class Foo which depends on class Bar. And my app tries to compile Foo before Bar is compiled, thus during compilation of Foo I can implement a logic to generate the source code of Bar and compile it and then continue compilation of Foo. Is this kind of stuff doable with ForwardJavaFileManager ? – Aholla

You might be able to implement that type of thing on top of what I describe. I do something kind of similar. If I get certain errors, I use regex to find out what is missing and generate/compile a stub for it. But if you're saying that JDT gives you hooks right in the middle of compilation to find missing symbols, I don't think Javac has that available. You either have to figure it out in advance, or with the error, or the list(...) call is for a package, so if you know that a request for a certain package means you have to generate something, then you could put in the cascade compile there. – Baron

hmm... probably not the way for me to go. The point is when I compiling B, I found A is missing, not only the class A is missing, but the source code is also not ready, I need to generate the source code A out from a template file [a]. JDT's findType call fits perfect for the needs. – Aholla

Yes, I'm in the same situation. But with Javac, I have to examine the error after trying to compile B, discover that A is missing, generate code for A, compile A, then recompile B. If JDT gives you a hook right in the middle of compilation to resolve dependencies, that would be cleaner. – Baron

definitely, I don't have to catch the exception and do a lot of work then come back to recompile B. The workflow with JDT becomes compile A -> looking for B -> if B is not ready, then generate and compile B -> continue compilation of A. – Aholla

I grant the bounty to you as this is the closest answer :) – Aholla

Answer 2

Try reading through this HelloWorld example to see if it solves your problem. Without posting the code, it's hard to say what your specific issue is.

Tradesman

Here is the case. Suppose you have 2 string of source code to be compiled. One is "package foo; public class One {...}" and the other is "package bar; import foo.One; public class Two {...}". Everything is okay until it starts to compile the second class "Two". Compiler will report that foo.One could not be located. – Aholla

I see. I'll leave this up and look for a better solution. It sounds like you need some sort of URL classloader. – Tradesman

So I give up the standard sun JDT compiler and end up with ECJ. Do you have any new findings? – Aholla