Sharing dynamically loaded classes with JShell instance

Question

Please view the edits below

I'm trying to create a JShell instance that gives me access to, and lets me interact with objects in the JVM it was created in. This works fine with classes that have been available at compile time but fails for classes that are loaded dynamically.

public class Main {

    public static final int A = 1;
    public static Main M;

    public static void main(String[] args) throws Exception {
        M = new Main();
        ClassLoader cl = new URLClassLoader(new URL[]{new File("Example.jar").toURL()}, Main.class.getClassLoader());
        Class<?> bc = cl.loadClass("com.example.test.Dynamic");//Works
        JShell shell = JShell.builder()
                .executionEngine(new ExecutionControlProvider() {
                    @Override
                    public String name() {
                        return "direct";
                    }

                    @Override
                    public ExecutionControl generate(ExecutionEnv ee, Map<String, String> map) throws Throwable {
                        return new DirectExecutionControl();
                    }
                }, null)
                .build();
        shell.eval("System.out.println(com.example.test.Main.A);");//Always works
        shell.eval("System.out.println(com.example.test.Main.M);");//Fails (is null) if executionEngine is not set
        shell.eval("System.out.println(com.example.test.Dynamic.class);");//Always fails
    }
}

Additionally, exchanging DirectExecutionControl with LocalExecutionControl gives the same results, but I do not understand the difference between the two classes.

How would I make classes that are loaded at runtime available to this JShell instance?

Edit: The first part of this question has been solved, below is the updated source code to demonstrate the second part of the issue

public class Main {

    public static void main(String[] args) throws Exception {
        ClassLoader cl = new URLClassLoader(new URL[]{new File("Example.jar").toURL()}, Main.class.getClassLoader());
        Class<?> c = cl.loadClass("com.example.test.C");
        c.getDeclaredField("C").set(null, "initial");
        JShell shell = JShell.builder()
                .executionEngine(new ExecutionControlProvider() {
                    @Override
                    public String name() {
                        return "direct";
                    }

                    @Override
                    public ExecutionControl generate(ExecutionEnv ee, Map<String, String> map) throws Throwable {
                        return new DirectExecutionControl();
                    }
                }, null)
                .build();
        shell.addToClasspath("Example.jar");
        shell.eval("import com.example.test.C;");
        shell.eval("System.out.println(C.C)"); //null
        shell.eval("C.C = \"modified\";");
        shell.eval("System.out.println(C.C)"); //"modified"
        System.out.println(c.getDeclaredField("C").get(null)); //"initial"
    }
}

This is the expected output, if the JVM and the JShell instance do not share any memory, however adding com.example.test.C directly to the project instead of loading it dynamically changes the results as follows:

shell.eval("import com.example.test.C;");
shell.eval("System.out.println(C.C)"); //"initial"
shell.eval("C.C = \"modified\";");
shell.eval("System.out.println(C.C)"); //"modified"
System.out.println(c.getDeclaredField("C").get(null)); //"modified"

Why is the memory between the JVM and the JShell instance not shared for classes loaded at runtime?

EDIT 2: The issue seems to be caused by different class loaders

Executing the following code in the context of the above example:

System.out.println(c.getClassLoader()); //java.net.URLClassLoader
shell.eval("System.out.println(C.class.getClassLoader())"); //jdk.jshell.execution.DefaultLoaderDelegate$RemoteClassLoader
shell.eval("System.out.println(com.example.test.Main.class.getClassLoader())"); //jdk.internal.loader.ClassLoaders$AppClassLoader

This shows, that the same class, com.example.test.C is loaded by two different classloaders. Is it possible to add the class to the JShell instance without loading it again? If no, why is the statically loaded class already loaded?

Rastus

which IDE did you use to run your code ? – Ng

I tried both Netbeans and directly running it. – Rastus

is there an end-goal to this, admittedly, very interesting exercise? – Leopoldoleor

Well, it's actually working now, see the my answer below. I did it to debug Minecraft server plugins, but it was mostly just that, an interesting exercise. I hope the solution is helpful for others that want to understand how the JShell interacts with the JVM that launched it. – Rastus

Answer 1

The solution is to create a custom LoaderDelegate implementation, that supplies instances of already loaded classes instead of loading them again. A simple example is to use the default implementation, DefaultLoaderDelegate (source) and override the findClass method of its internal RemoteClassLoader

@Override
protected Class<?> findClass(String name) throws ClassNotFoundException {
    byte[] b = classObjects.get(name);
    if (b == null) {
        Class<?> c = null;
        try {
            c = Class.forName(name);//Use a custom way to load the class
        } catch(ClassNotFoundException e) {
        }
        if(c == null) {
            return super.findClass(name);
        }
        return c;
    }
    return super.defineClass(name, b, 0, b.length, (CodeSource) null);
}

To create a working JShell instance, use the following code

JShell shell = JShell.builder()
    .executionEngine(new ExecutionControlProvider() {
        @Override
        public String name() {
            return "name";
        }

        @Override
        public ExecutionControl generate(ExecutionEnv ee, Map<String, String> map) throws Throwable {
            return new DirectExecutionControl(new CustomLoaderDelegate());
        }
    }, null)
    .build();
shell.addToClasspath("Example.jar");//Add custom classes to Classpath, otherwise they can not be referenced in the JShell

Rastus

Can you award the bounty to yourself? – Turbojet

As far as I know this is impossible – Rastus

see my answer, it's easier – Isidore

Answer 2

only speaking to a small part of this rather substantial question:

Additionally, exchanging DirectExecutionControl with LocalExecutionControl gives the same results, but I do not understand the difference between the two classes

LocalExecutionControl extends DirectExecutionControl and it overrides only invoke(Method method), the bodies of which are ...

local:

    Thread snippetThread = new Thread(execThreadGroup, () -> {
            ...
            res[0] = doitMethod.invoke(null, new Object[0]);
            ...
    });

direct:

    Object res = doitMethod.invoke(null, new Object[0]);

so the difference between the two classes is that direct invokes the method in the current thread, and local invokes it in a new thread. the same classloader is used in both cases, so you'd expect the same results in terms of sharing memory and loaded classes

Lugsail

Answer 3

Now, there is better and easier solution:

package ur.pkg;

import jdk.jshell.JShell;
import jdk.jshell.execution.LocalExecutionControlProvider;

public class TestShell {
    public static int testValue = 5;

    public static void main(String[] args) {
        JShell shell = JShell.builder().executionEngine(new LocalExecutionControlProvider(), null).build();
        TestShell.testValue++;
        System.out.println(TestShell.testValue);
        shell.eval("ur.pkg.TestShell.testValue++;").forEach(p -> {
            System.out.println(p.value());
        });
        System.out.println(TestShell.testValue);

    }

}

Default execution engine is JDI, but u can switch it to local or own.

Isidore

For me this prints: 6, null, 6. JDK 9.0.4. Do I need to use a specific Java version? – Rastus