Compile Groovy class at runtime in Java

Question

I am successfully able to compile Groovy in Java at runtime and store it in a database and pull it out. I can't compile a Groovy class if it has inner classes or an inner enum. Has anyone successfully compiled Groovy code like this and included inner classes/enums and able to pull the script out by classname?

For example, I want to load the "Test" script shown below that contains inner classes and run the script at run time.

Compiler code:

public byte[] compileGroovyScript(final String className, final String script) {
    byte[] compiledScriptBytes = null;
    CompilationUnit compileUnit = new CompilationUnit();
    compileUnit.addSource(className, script);
    compileUnit.compile(Phases.CLASS_GENERATION);

    for (Object compileClass : compileUnit.getClasses()) {
        GroovyClass groovyClass = (GroovyClass) compileClass;
        compiledScriptBytes = groovyClass.getBytes();
    }

    return compiledScriptBytes;
}

Code to pull script out:

public Class getGroovyScript(final String className, final byte[] script) {
    Class clazz = null;

    try (GroovyClassLoader classLoader = new GroovyClassLoader(this.getClass().getClassLoader())) {
        clazz = classLoader.defineClass(className, script);
    } catch (IOException e) {
    } catch (Exception e) {
    }

    return clazz;
}

Code to run the script:

Class groovyClass = app.getGroovyScript(className, compiledScript);
TestScript script = (TestScript) groovyClass.newInstance();
System.out.println(script.getMessage());

Groovy script:

import com.groovy.groovy.TestScript

class Test implements TestScript {

    String getMessage() {
        [1..10].each(){
            println it
        }
        return "Jello"
    }
}

Tinny

You iterate over classes from compilationUnit, but You return only bytes from the last class compiledScriptBytes = groovyClass.getBytes(); I don't know if this is the case, but this looks like a potential bug. – Reede

Well I tried iterating over all the classes and storing them in one byte[] but that did not work when getting the groovy class and casting it to my Java interface. – Tinny

Answer 1

It isn't clear from the description why you are doing the compiling yourself. If you can just let Groovy do it for you then the whole thing can just be simplified to something like this:

String script = // string containing the script you want to parse

GroovyClassLoader groovyClassLoader = new GroovyClassLoader();
Class theParsedClass = groovyClassLoader.parseClass(script);

Tranquilize

I mentioned that I wanted to compile the Groovy scripts and put them in the database. The reason for this is for performance, so when I want to run a script I don't have to compile it multiple times. The scripts are run every minute and compiling a script every minute seems inefficient. – Tinny

If I didn't have to compile the scripts and store them, this would work. – Tinny

If every minute you are retrieving bytes from a database and then calling defineClass() to turn those bytes into a Class then you can probably improve on that. Is your application such that there is some reason that you cannot access a class loader which is available each time you need access to the script class? As long as you can do that you can avoid having to do the defineClass() thing to turn your bytes into a Class every time you need to run it. You could do it just once and then every time you need to run that script, the Class would already be available in the class loader. – Tranquilize

The compiling of the scripts is done on a web support tool for engineers that is run in a separate JVM than the application that actually runs the scripts. – Tinny

It still sounds like you are asking the class loader to reload the class from byte[] each time you need to run the script. In any case, that isn't the problem you asked about so I won't bother you with it any longer. Sorry for the distraction. Do either of the solutions I mentioned above address the concern that you originally asked about? – Tranquilize

Well this one worked if I wasn't compiling the scripts ahead of time. The others failed as it still didn't compile the inner classes. I need to compile the whole Groovy script even if it has inner classes and store it in the database as a byte array. Then I need to extract that byte array from the database and run the script. – Tinny

Your original question says "Has anyone successfully compiled Groovy code like this and included inner classes/enums?". The code I showed does compile Groovy code which includes inner classes. If you want to compile the inner classes then iteratove over all of the classes returned from compileUnit.getClasses() and process each of those byte[]. A problem with your original code is you are iterating over all of those but only dealing with the last one. Look at the for loop in your compileGroovyScript. – Tranquilize

After looking back at this we found flaws with storing compiled Groovy scripts in the database. If Groovy gets updated in our project then all the scripts in the database need to be recompiled. Another case was the implementation I wrote it wasn't made possible to easily handle inner classes, enums, etc. This is a better way to compile scripts at runtime. The only issue I have right now is I am a ton of dead GroovyClassLoaders in Perm Gen that aren't being cleaned up by Full GC (#27451558). – Tinny

Answer 2

Ok this may be a little late but hopefully it helps the next person. I think you need to save a List for each groovy class and then cl.defineClass and finally cl.loadClass. I think groovy sometimes compile to a list of classes basically as in below when I addSource(), I add one class and then loop over all the generated classes from that one file.

This is the code I am currently running(though I have not tried saving and reloading at a later time)

    GroovyClassLoader cl = new GroovyClassLoader();
    CompilationUnit compileUnit = new CompilationUnit();
    compileUnit.addSource(scriptCode.getClassName(), scriptCode.getScriptSourceCode());
    compileUnit.compile(Phases.CLASS_GENERATION);
    compileUnit.setClassLoader(cl);

    GroovyClass target = null;
    for (Object compileClass : compileUnit.getClasses()) {
        GroovyClass groovyClass = (GroovyClass) compileClass;
        cl.defineClass(groovyClass.getName(), groovyClass.getBytes());
        if(groovyClass.getName().equals(scriptCode.getClassName())) {
            target = groovyClass;
        }
    }

    if(target == null) 
        throw new IllegalStateException("Could not find proper class");

    return cl.loadClass(target.getName());

take note of the cl.defineClass call which puts the class in the classloader so when it is looked up(the enum or innerclass), it will be there.

and so now I think you do not need to create your own class loader(though you avoid useless defineClass until it is needed with your own classloader which can be useful and more performant).

Agonize

Answer 3

This forgoes any error handling for the sake of simplicity here, but this is probably what you want:

public byte[] compileGroovyScript(final String className, final String script) {
    byte[] compiledScriptBytes = null;
    CompilationUnit compileUnit = new CompilationUnit();
    compileUnit.addSource(className, script);
    compileUnit.compile(Phases.CLASS_GENERATION);

    List classes = compileUnit.getClasses();
    GroovyClass firstClass = (GroovyClass)classes.get(0);
    compiledScriptBytes = firstClass.getBytes();

    return compiledScriptBytes;
}

Tranquilize

Answer 4

Depending on your requirements, you might want to provide access to the inner classes and you could do that with something like this which finds the class with the matching name instead of assuming the first class:

public byte[] compileGroovyScript(final String className, final String script) {
    byte[] compiledScriptBytes = null;
    CompilationUnit compileUnit = new CompilationUnit();
    compileUnit.addSource(className, script);
    compileUnit.compile(Phases.CLASS_GENERATION);

    for (Object compileClass : compileUnit.getClasses()) {
        GroovyClass groovyClass = (GroovyClass) compileClass;
        if(className.equals(groovyClass.getName())) {
            compiledScriptBytes = groovyClass.getBytes();
            break;
         }

    }

    return compiledScriptBytes;
}

Tranquilize

Answer 5

I am running into this myself but having just done an on-demand java compiler at runtime, I believe you are running into the same issue I solved in this code

https://github.com/deanhiller/webpieces/tree/master/runtimecompile/src/main/java/org/webpieces/compiler/api

webpieces/runtimecompile is a re-usable on-demand java compiler using the eclipse compiler.

Now, for groovy, I think you are running into this case

1. you compile ONE script
2. this results in 'multiple' class file objects (I think) just like mine did
3. This is where you need to store EACH in the database SEPARATELY
4. Then you need a classloader that tries to lookup the 'inner classes' when jvm asks for it
5. finally you do a yourclassLoader.loadApplicationClass (much like the one in CompileOnDemandImpl.java in the project above
6. To be clear, step 5 causes step 4 to happen behind the scenes (and that is what is confusing).

If you step through the test case AnonymousByteCacheTest, it pretty much is doing something like that.

you don't need to install ANYTHING to run the build on that project, just clone it and "./gradlew test" and will pass and "./gradlew eclipse" or "./gradlew idea" and it generates IDE files so you can step through it.

It is very very similar. I am trying to get the groovy version working next myself.

Agonize