Why does my custom SecurityManager cause exceptions the 16th time I create an object with Constructor.newInstance?

Question

I am currently working on developing a small Java application in which trusted code must be run alongside untrusted code. To accomplish this, I have installed a custom SecurityManager that throws SecurityExceptions any time a permission is checked.

As a bridge between the trusted and untrusted code, I have a thread that uses Constructor.newInstance() to instantiate an object of an untrusted type. At the time that it makes this call, the security manager is configured to block everything. Interestingly, the first 15 times that I try to create objects using Constructor.newInstance(), everything works fine, but the 16th time I get a SecurityException.

I've managed to get this down to a simple test program:

import java.lang.reflect.*;
import java.security.*;

public class Main {
    /* Track how many instances have been created so that we can see when the exception
     * is thrown.
     */
    private static int numInstances = 0;
    public Main() {
        System.out.println("Number created: " + ++numInstances);
    }

    public static void main(String[] args) {
        /* Get the constructor for Main so that we can instantiate everything
         * later on.
         */
        Constructor<Main> ctor;
        try {
            ctor = Main.class.getConstructor();
        } catch (NoSuchMethodException e) {
            e.printStackTrace();
            return;
        }

        /* Install a super prohibitive security manager that disallows all operations. */
        System.setSecurityManager(new SecurityManager() {
            @Override
            public void checkPermission(Permission p) {
                /* Nothing is allowed - any permission check causes a     security
                 * exception.
                 */
                throw new SecurityException("Not permitted: " + p);
            }
        });

        /* Continuously create new Main objects. */
        try {
            while (true) {
                ctor.newInstance();
            }
        } catch (Exception e) {
            e.printStackTrace();
            return;
        }
    }
}

This program installs a SecurityManager whose checkPermission always throws an exception regardless of what permission is requested. It then sits in a loop and uses ctor.newInstance() to instantiate a harmless Main object that prints out the number of instances generated so far. The output of this program, on my system, is as follows:

Number created: 1
Number created: 2
Number created: 3
Number created: 4
Number created: 5
Number created: 6
Number created: 7
Number created: 8
Number created: 9
Number created: 10
Number created: 11
Number created: 12
Number created: 13
Number created: 14
Number created: 15
java.lang.SecurityException: Not permitted: ("java.lang.RuntimePermission" "createClassLoader")
    at Main$1.checkPermission(Main.java:32)
    at java.lang.SecurityManager.checkCreateClassLoader(SecurityManager.java:611)
    at java.lang.ClassLoader.checkCreateClassLoader(ClassLoader.java:274)
    at java.lang.ClassLoader.<init>(ClassLoader.java:316)
    at sun.reflect.DelegatingClassLoader.<init>(ClassDefiner.java:72)
    at sun.reflect.ClassDefiner$1.run(ClassDefiner.java:60)
    at sun.reflect.ClassDefiner$1.run(ClassDefiner.java:58)
    at java.security.AccessController.doPrivileged(Native Method)
    at sun.reflect.ClassDefiner.defineClass(ClassDefiner.java:57)
    at sun.reflect.MethodAccessorGenerator$1.run(MethodAccessorGenerator.java:399)
    at sun.reflect.MethodAccessorGenerator$1.run(MethodAccessorGenerator.java:396)
    at java.security.AccessController.doPrivileged(Native Method)
    at sun.reflect.MethodAccessorGenerator.generate(MethodAccessorGenerator.java:395)
    at sun.reflect.MethodAccessorGenerator.generateConstructor(MethodAccessorGenerator.java:94)
    at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:48)
    at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
    at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
    at Main.main(Main.java:39)

According to the Javadoc for RuntimePermission, the createClassLoader permission is a risky one to grant:

This is an extremely dangerous permission to grant. Malicious applications that can instantiate their own class loaders could then load their own rogue classes into the system. These newly loaded classes could be placed into any protection domain by the class loader, thereby automatically granting the classes the permissions for that domain.

I have two questions:

1. What specifically is causing this error? Why is it that on the 16th time around, I get a request for a classloader? I suspect this has to do with Java trying to optimize the reflection by generating bytecode to directly instantiate the object, but I'm not sure.

2. Without whitelisting the createClassLoader privilege, which is dangerous, is there a way to instantiate the untrusted objects from trusted code?

3. Am I fundamentally approaching this the wrong way?

Thanks!

Conyers

I tried your code, same behavior. If you remove the throw exception and add a print statement, the checkPermission gets called 3 times before creating the 16th instance. – Augusto

Answer 1

Check out this at GrepCode:

72  private static int     inflationThreshold = 15;

15 is the default value for the inflation threshold, the count of reflective calls before a more aggressive optimization is introduced in NativeConstructorAccessorImpl:

47  if (++numInvocations > ReflectionFactory.inflationThreshold()) {
48      ConstructorAccessorImpl acc = (ConstructorAccessorImpl)
49          new MethodAccessorGenerator().
50              generateConstructor(c.getDeclaringClass(),
51                                  c.getParameterTypes(),
52                                  c.getExceptionTypes(),
53                                  c.getModifiers());
54      parent.setDelegate(acc);

And that particular code causes a new class loader to be instantiated, resulting in your exception at the 16th iteration. Bytecode generation happens in the MethodAccessorGenerator class, and this is the most interesting bit:

387  // Load class
388  vec.trim();
389  final byte[] bytes = vec.getData();
390  // Note: the class loader is the only thing that really matters
391  // here -- it's important to get the generated code into the
392  // same namespace as the target class. Since the generated code
393  // is privileged anyway, the protection domain probably doesn't
394  // matter.
395  return AccessController.doPrivileged(
396      new PrivilegedAction<MagicAccessorImpl>() {
397          public MagicAccessorImpl run() {
398                  try {
399                  return (MagicAccessorImpl)
400                  ClassDefiner.defineClass
401                          (generatedName,
402                           bytes,
403                           0,
404                           bytes.length,
405                           declaringClass.getClassLoader()).newInstance();
406                  } catch (InstantiationException e) {
407                      throw (InternalError)
408                          new InternalError().initCause(e);
409                  } catch (IllegalAccessException e) {
410                      throw (InternalError)
411                          new InternalError().initCause(e);
412                  }
413              }
414          });

As for granting that permission, you do still have the choice of carefully forming a protection domain for your code, to which you grant the permission, without granting it to foreign code.

Nagel

Detective Marko Topolnik, on the case! – Augusto

A temptation I couldn't possibly have resisted :) – Nagel

This is fantastic. I'm still learning about protection domains and apologize if this is a silly question, but how would I construct the domain to prevent the untrusted code from using classloaders while letting my trusted code use it for the newInstance call? Or would putting the whole thing into a protection domain where classloader privileges are revoked cause the internal implementation not to try to inflate? – Conyers

Unfortunately, I have spent this time thinking about it and I don't know how to grant the necessary privileges to the JDK reflective code without transitively granting to the eventually invoked constructor. You may perhaps disable inflation altogether. – Nagel

The Java library does randomly need permissions to run. Typically t's not worth having a "custom" security manager since 1.2 (1998). Instead checks get forward through to java.security.AccessController. Looking the example code your SecurityManager probably wouldn't work as it doesn't appear to be trusted. Any check that passes through it is going to fail because it is not privileged. Unsurprisingly, code that manages trust needs to be trusted (to at least the level of the trust). – Emerald

@TomHawtin-tackline How can the check fail if it passes through the SecurityManager? Isn't it the only authority on making security checks fail? – Nagel

@MarkoTopolnik Since 1.2 the SecurityManager does little more than provide poorly thought-out convenience methods to forward on security check requests. – Emerald

@TomHawtin-tackline But if you override the main entry-point method, as OP has done, then all that forwarding is gone, isn't it? – Nagel

@MarkoTopolnik The OP has overridden the main exit-point to always fail, which isn't very useful. If they were to try to forward to the standard implementation (as indeed they do in their follow-up question), then it still always fails, which still isn't very useful. – Emerald