Why 'T.super.toString()' and 'super::toString' use a synthetic accessor method?

Question

Consider the following set of expressions:

class T {{
/*1*/   super.toString();      // direct
/*2*/   T.super.toString();    // synthetic
        Supplier<?> s;
/*3*/   s = super::toString;   // synthetic
/*4*/   s = T.super::toString; // synthetic
}}

Which gives the following result:

class T {
    T();
     0  aload_0 [this]
     1  invokespecial java.lang.Object() [8]
     4  aload_0 [this]
     5  invokespecial java.lang.Object.toString() : java.lang.String [10]
     8  pop           // ^-- direct
     9  aload_0 [this]
    10  invokestatic T.access$0(T) : java.lang.String [14]
    13  pop           // ^-- synthetic
    14  aload_0 [this]
    15  invokedynamic 0 get(T) : java.util.function.Supplier [21]
    20  astore_1 [s]  // ^-- methodref to synthetic
    21  aload_0 [this]
    22  invokedynamic 1 get(T) : java.util.function.Supplier [22]
    27  astore_1      // ^-- methodref to synthetic
    28  return

    static synthetic java.lang.String access$0(T arg0);
    0  aload_0 [arg0]
    1  invokespecial java.lang.Object.toString() : java.lang.String [10]
    4  areturn

    Bootstrap methods:
    0 : # 40 invokestatic java/lang/invoke/LambdaMetafactory.metafactory:...
        #43 invokestatic T.access$0:(LT;)Ljava/lang/String;
    1 : # 40 invokestatic java/lang/invoke/LambdaMetafactory.metafactory:...
        #46 invokestatic T.access$0:(LT;)Ljava/lang/String;
}

Why java code lines /*2*/, /*3*/ and /*4*/ produce and use a synthetic accessor method access$0? I would expect the line /*2*/ and bootstrap methods for lines /*3*/ and /*4*/ to also use invokespecial as the line /*1*/ does.

Especially when the method Object::toString is accessible directly from the relevant scope, e.g. the following method reference doesn't wrap a call to a synthetic accessor method:

class F {{
    Function<Object, ?> f = Object::toString; // direct
}}

However, there is a difference:

class O {{
        super.toString();      // invokespecial -> "className@hashCode"
        O.super.toString();    // invokespecial -> "className@hashCode"
        Supplier<?> s;
        s = super::toString;   // invokespecial -> "className@hashCode"
        s = O.super::toString; // invokespecial -> "className@hashCode"
        Function<Object, ?> f = Object::toString;
        f.apply(O.super); // invokeinterface -> "override"
    }
    public String toString() {return "override";}
}

Which brings another question: Is there a way how to bypass an override in ((Function<Object, ?> Object::toString)::apply?

Aron

Note that the behavior of a lambda expression is fixed and can't be altered by the invoker. Hence, the invalid (but accepted by Eclipse) syntax f.apply(O.super); can't make a difference to f.apply(O.this); as it's the same object and the invocation behavior is fixed for this function. You can't create a Function<Object, ?> that ignores overrides (with legal Java constructs), but you can create a Function<O, ?> that ignores overrides, using a helper method, similar to these synthetic access$n methods. – Frock

Tested. A ((Function<O, ?>) O::helper).apply(this) where private String helper() {return super.toString();} works fine. However, it works only 1 level up, and unless you create a chain of helpers up the hierarchy, you'll never get a hold of the real Object::toString, right? Thanks anyway. – Aron

For inner classes, when the helper is private String helper() {return OuterMostClass.super.toString();} it will still only call the parent of the OuterMostClass, not Object, so it seems there is no plain java way. – Aron

...where a simple invokespecial would do... – Aron

No, that was long ago, when invokespecial allowed to skip/target arbitrary classes (Java 1.0). Nowadays, for non-private, non-interface methods the target type must be the direct superclass of the containing class. Otherwise, the verifier is entitled to reject it. There was a time when the absence of the ACC_SUPER flag could reinforce the old behavior, but the most recent JVMs treat all classes like if the flag is present. – Frock

Answer 1

An invocation of the form super.method() allows to bypass an overriding method() in the same class, invoking the most specific method() of the super class hierarchy. Since, on the byte code level, only the declaring class itself can ignore its own overriding method (and potential overriding methods of subclasses), a synthetic accessor method will be generated if this kind of invocation should be performed by a different (but conceptionally entitled) class, like one of its inner classes, using the form Outer.super.method(...), or a synthetic class generated for a method reference.

Note that even if a class doesn't override the invoked method and it seems to make no difference, the invocation has to be compiled this way as there could be subclasses at runtime overriding the method and then, it will make a difference.

It's interesting that the same thing happens when using T.super.method() when T actually isn't an outer class but the class containing the statement. In that case, the helper method isn't really necessary, but it seems that the compiler implements all invocations of the form identifier.super.method(...) uniformly.

---

As a side note, Oracle's JRE is capable of circumventing this byte code restriction when generating classes for lambda expressions/method references, thus, the accessor methods are not needed for method references of the super::methodName kind, which can be shown as follows:

import java.lang.invoke.*;
import java.util.function.Supplier;

public class LambdaSuper {
    public static void main(String[] args) throws Throwable {
        MethodHandles.Lookup l=MethodHandles.lookup();
        MethodType mt=MethodType.methodType(String.class);
        MethodHandle target=l.findSpecial(Object.class, "toString", mt, LambdaSuper.class);
        Supplier<String> s=(Supplier)LambdaMetafactory.metafactory(l, "get",
            MethodType.methodType(Supplier.class, LambdaSuper.class),
            mt.generic(), target, mt).getTarget().invokeExact(new LambdaSuper());
        System.out.println(s.get());
    }

    @Override
    public String toString() {
        return "overridden method";
    }
}

The generated Supplier will return something alike LambdaSuper@6b884d57 showing that it invoked the overridden Object.toString() method rather than the overriding LambdaSuper.toString(). It seems that the compiler vendors are careful regarding what to expect from the JRE capabilities and, unfortunately, this part seems to be a bit underspecified.

Still, for real inner class scenarios, they are required.

Frock

Interestingly, my test with javac revealed that it even generates two identical accessor methods for T.super.toString(), one being entirely unused. – Frock

Well, Eclipse and javac give totally different results. Eclipse emits 1 accessor and 2 methodrefs to that accessor. While javac emits 2 accessors (one called directly, the other from lambda), and 2 lamda methods, of which one calls an accessor, the other does invokespecial on Object::toString directly, without an accessor. – Aron

To your answer -- I can accept the need for an accessor in a super class (to bypass an override), but why is it here in the class itself? Since super.toString() uses invokespecial, not an accessor, why all the other don't do the same? – Aron

Well, javac generates multiple accessor methods, because it adds line debug information to it. For the reason, why it doesn't treat T.super like super when T is the current class, the uncomfortable truth is that compilers not always generate the most efficient code. – Frock

For other examples of less than optimal compiler output see Why switch on String compiles into two switches or try with resources introduce unreachable bytecode – Frock

Re 1st paragraph: I think the last sentence should stress that it applies only for invocations on an outer class, that is in the form T.super.method(). – Aron

Re T.super.method() when invoked in T: It seems that a chain of accessors is generated up to the denoted type. In the outer most type accessor, in T, the actual method call is contained, while intermediate accessors serve for passing down reference to an immediate outer class. So if T is the current class, it's a degenerated case where only the final accessor with a method call will be created. – Aron

For the form super::toString, the accessor is created for a completely different reason -- I believe it's a closure capturing the super instance, since it's an instance method reference. So this case has nothing to do with bypassing an override. – Aron

And the last case, T.super::toString is a combination of the two. :o) – Aron

Thanks for the edit, now it's indeed more clear, so I casted an upvote. However, unless my recent findings are wrong, your answer explains only 1.5 of the 3 cases. I'll compile an answer myself and will wait for other readers to react. – Aron

There is no such thing as a "super instance". Whether you write this::toString or super::toString, in both cases, the same instance is captured, only the type of invocation changes. But this::toString doesn't need an accessor method. Note that the example code in my answer does exactly the same as super::toString, capturing the instance, but without needing an accessor method. – Frock

Which case do you think to be unanswered? – Frock

But, as you wrote, the difference is in invocation type (invokevirtual vs invokespecial). By super instance I meant invokespecial on a parent type (with this as a target, agreed). So does that Oracle code example translate into invokespecial directly inside a bootstrap method? – Aron

Eclipse translates super::toString into a methodref to an "accessor" that just takes this as an argument, while () -> super.toString() it translates to a lamda method, which uses this directly. OpenJDK uses in both cases the latter. – Aron

Yeah, OpenJDK prefers to always use static methods for lambda expressions, even if this is captured. Since methods hosting the code of explicit lambda expressions are private, that's a minor difference, as it will always be a non-virtual call. And that should answer the other question, yes, the generated classes are capable of doing non-virtual calls, including private methods, and in the example, Oracle generates a class which does invokespecial with the intended super call semantic directly. – Frock

Re "what was unanswered": Explanation of lines /*3*/ and /*4*/ as closures being the reason for a synthetic method, especially in the case of super::toString, which is not a case of an accessor chain to an outer class. Still I'd like to hear if invokespecial is legal in a bootstrap method (as suggested by the Oracle example). – Aron

Actually, that's the opposite: OpenJDK is not using static methods (that I meant by "uses this directly"), so it is virtual. That's Eclipse who uses a static method, but just in case of a methodref, not for a lambda. – Aron

Being closures is not the reason. As said, this::method is a closure as well and doesn't need such a method. It's all about the accessibility and invocation type. invokespecial is always legal if you're invoking a private method of your own class, as all lambda expressions are compiled into private synthetic methods. Regarding super invocations, as said, it's a bit underspecified. – Frock

OK, so at least a "part of the reason"? That this::method goes without a need for a synthetic method is apparent. And super.method() goes also without. And for Oracle, also super::method goes without. So the invokespecial can go directly to a bootstrap method (lambda metafactory) and both Eclipse and OpenJDK could also go without an accessor, right? So the reason would be "Eclipse, OpenJDK", not "closure". – Aron

Last question, hopefully: How does Oracle act upon () -> this.method()? It should create a lambda method, at last, shouldn't it? – Aron

() -> this.method() is always compiled into a lambda method. It would be possible to recognize this pattern and replace it by an equivalent method reference, but 1.) someone has to implement it and create test cases, etc. and 2.) this hypothetical optimization should not apply when line debug attributes are turned on (they are by default) as the source code line number(s) of the lambda expression get attached to that synthetic lambda method. – Frock

Answer 2

Holger already explained why it is happening — super reference is restricted to the immediate child class only. Here's just a more verbose version of what's really happening there:

---

Call to an enclosing type's super class' method

class T {
    class U {
        class V {{
/*2*/       T.super.toString();
        }}
    }
}

It generates a chain of synthetic accessor methods:

class T {
    static synthetic String access$0(T t) { // executing accessor
        return t.super.toString(); // only for the refered outer class
    }
    class U { // new U(T.this)
        static synthetic T access$0(U u) { // relaying accessor
            return T.this; // for every intermediate outer class
        }
        class V {{ // new V(U.this)
            T.access$0(U.access$0(U.this)); // T.access$0(T.this)
        }}
    }
}

When T is the immediately enclosing class, i.e. there are no intermediate outer classes, only the "executing" accessor is generated in class T (i.e. in itself, which seems to be unnecessary).

N.B.: The accessor chain is generated by Eclipse, but not by OpenJDK, see bellow.

---

Method reference to own super class' method

class T {
    class U {
        class V {{
            Supplier<?> s;
/*3*/       s = super::toString;
        }}
    }
}

This generates a synthetic accessor method and a bootstrap method delegating to it:

class T {
    class U {
        class V {
            static synthetic String access$0(V v) {
                return v.super.toString();
            }
            dynamic bootstrap Supplier get(V v) { // methodref
                return () -> V.access$0(v); // toString() adapted to get()
            }
            {
                get(V.this);
            }
        }
    }
}

It's a singular case similar to the previous one, since super::toString is here equivalent to V.super::toString, so the synthetic accessor is generated in the class V itself. A new element here is the bootstrap method for adapting Object::toString to Supplier::get.

N.B.: Here only OracleJDK is "smart" enough (as Holger pointed out) to avoid the synthetic accessor by placing the super call directly into the method reference adapter.

---

Method reference to an enclosing type's super class' method

class T {
    class U {
        class V {{
            Supplier<?> s;
/*4*/       s = T.super::toString;
        }}
    }
}

As you can expect, this is a combination of the two previous cases:

class T {
    static synthetic String access$0(T t) { // executing accessor
        return t.super.toString(); // only for the refered outer class
    }
    class U { // new U(T.this)
        static synthetic T access$0(U u) { // relaying accessor
            return T.this; // for every intermediate outer class
        }
        class V { // new V(U.this)
            dynamic bootstrap Supplier get(T t) { // methodref
                return () -> T.access$0(t); // toString() adapted to get()
            }
            {
                get(U.access$0(U.this)); // get(T.this)
            }
        }
    }
}

Nothing really new here, just note that inner class always receives only the instance of the immediate outer class, so in the class V, using T.this it either might go through the whole chain of intermediate synthetic accessor methods, e.g. U.access$0(V.U_this) (as in Eclipse), or take the advantage of package visibility of these synthetic fields (that reference outer.this) and translate T.this to V.U_this.T_this (as in OpenJDK).

---

N.B.: The above translations are as per Eclipse compiler. OpenJDK differs in generating instance synthetic lambda methods for method references, instead of static synthetic accessor methods as Eclipse does, and also avoids the accessor chain, so in the last case OpenJDK emits:

class T {
    static synthetic String access$0(T t) { // executing accessor
        return t.super.toString(); // only for the refered outer class
    }
    class U { // new U(T.this)
        class V { // new V(U.this)
            instance synthetic Object lambda$0() {
                return T.access$0(V.U_this.T_this); // direct relay
            }
            dynamic bootstrap Supplier get(V v) { // methodref
                return () -> V.lambda$0(v); // lambda$0() adapted to get()
            }
            {
                get(V.this);
            }
        }
    }
}

---

To sum up, it's quite dependent on the compiler vendor.

Aron