On-the-fly, in-memory java code compilation for Java 5 and Java 6

Question

How can I compile java code from an arbitrary string (in memory) in Java 5 and Java 6, load it and run a specific method on it (predefined)?

Before you flame this, I looked over existing implementations:

• Most rely on Java 6 Compiler API.
• Those that don't, rely on tricks.
• Yes, I checked out commons-jci. Either I'm too dense to understand how it works, or it just doesn't.
• I could not find how to feed the compiler my current class path (which is quite huge).
• On the implementation that worked (in Java 6), I could not find how to correctly load inner classes (or inner anonymous classes).
• I'd quite like it if the entire thing was in-memory, as the thing runs on multiple environments.

I'm sure this has been solved before, but I can't find anything that looks even half-production quality on google (except jci, which, as I've said before, I haven't managed to use).

Edit:

• I looked over JavaAssist - I need inner classes, Java 5.0 language level support and compiling with the entire classpath. Also, I'd like to create new classes on the fly. I might be mistaken, but I couldn't find how to do this with JavaAssit.
• I'm willing to use a file-system based solution (calling javac) but I don't know how to divine the classpath, nor how to later load the files (which are not in my classpath) with a special classloader that can be recycled for multiple invocations. While I do know how to research it, I'd prefer a ready solution.

Edit2: For now, I'm content with BeanShell "evaluate". Apparently it does everything I need it to (get a string, evaluate it in the context of the 'current' classpath. It does miss some of Java 5 features, but it can use enums (not define) and compiled 'generic' (erased) classes, so it should be enough for what I want.

I don't want to mark the answer as accepted yet since I do hope for a better solution to come up.

Edit3: Accepted the beanshell suggestion - it really works wonderfully.

Wynd

Answer 1

If you're not completely tied to compiling, solutions like Beanshell, groovy and the other scripting languages are easily embedded (in-fact, java has built-in support for plugging in a scripting language so your code doesn't even know what language the script is written in)

Beanshell should run any 100% java code IIRC, and I believe Groovy can run most java code--possibly all.

Tonietonight

This is a good solution, but I still hope for a better one with a real compile process. If nothing else crops up, I'll mark this as accepted. Thanks. – Wynd

You know, this API has always been available I think: openjdk.java.net/groups/compiler/doc/package-overview/… But it's not in the java. and therefore subject to change (although I don't think it has changed much) – Tonietonight

I'm not saying this is the correct solution for everyone, but it worked out perfectly for my needs. BSH 2.0 can really work wonders. – Wynd

Answer 2

JCI looks fine. This code snippet should be your base:

JavaCompiler compiler = new JavaCompilerFactory().createCompiler("eclipse");

MemoryResourceReader mrr = new MemoryResourceReader();
mrr.add("resource name string", yourJavaSourceString.getBytes());

MemoryResourceStore mrs = new MemoryResourceStore();

CompilationResult result = compiler.compile(sources, mrr, mrs);

// don't need the result, unless you care for errors/warnings
// the class should have been compiled to your destination dir

Any reason this should not work?

---

Edit: added a MemoryResourceStore to send the compiled class output to memory, like requested.

Also, setting javac settings, like classpath in your case, can be done via setCustomArguments(String[] pCustomArguments) in JavacJavaCompilerSettings class.

Madewell

No reason, but: 1. I can't find out how to divine the classpath. As I've said earlier, it's huge and quite complex. I'd love the compiler to use the classes "in memory" to compile. 2. I'll use file output as last resort, but I'd like to explore in-memory solutions first. – Wynd

Answer 3

You might want to check out Janino as well.

From their website:

Janino is a compiler that reads a JavaTM expression, block, class body, source file or a set of source files, and generates JavaTM bytecode that is loaded and executed directly. Janino is not intended to be a development tool, but an embedded compiler for run-time compilation purposes, e.g. expression evaluators or "server pages" engines like JSP.

http://www.janino.net/

Im currently using it in a pretty large mission critical project and it works just fine

Pongid

Forgot to mention. I did check out Janino. As I've said, I need 5.0 features in order to interoperate with existing code. – Wynd

Janino supports some 5.0 features, I just wasn't sure if it would cover those you needed. Good luck; I'd love to hear what you settle on. – Pongid

Janino seems far superior to BeanShell. Too bad about the 1.5 features. – Londoner

Answer 4

Javassist might interest you

Ore

I looked over JavaAssist, and it's not exactly what I'm looking for. I want to run code given to me as string, code that relies on my classes - I couldn't find a way to do this with JavaAssist. Also, the limitations of JavaAssist (no 5.0 features) are quite limiting for me. – Wynd

Answer 5

Run inside a web container like Tomcat and first generate a JSP page, and then invoke it.

This also allow you to get rid of the old class definitions by simply overwriting the JSP page instead of having your classloader slowly run full.

Is the "in-memory" requirement due to speed or due to not changing the code base?

Joppa

Actually, up until now I've used that method. It's clunky, but it works. But, I hate it, and the customers distrust it. I'd rather use a cleaner (or better hidden) approach. – Wynd

I'll bet you'll hate classloader problems issues more ;-) Be very careful what you do :) – Briarroot

Answer 6

ECJ Eclipse Java Compiler

Eclipse provides and uses its own compiler that is not javac

• The Eclipse compiler is used inside the IDE (Eclipse)
• The Eclipse compiler can also be used as a pure batch compiler outside of Eclipse

Compile a source file

$ java -jar ecj-3.5.2.jar HelloWorld.java

Delphiadelphic