Reflection - Method::getGenericReturnType no generic - visbility

Question

Description

I have an odd issue in which Method::getGenericReturnType() is not able to retrieve the generic type information.

Here is the minimized version:

public class Test {
    public static void main(String[] args) {
        Method method = B.class.getMethods()[0]; // foo() method inherited from A
        System.out.println(method.getGenericReturnType());
    }

    static class A {
        public List<String> foo() { return null; }
    }

    public static class B extends A {}
}

Output is

java.util.List

without any generic type information. This seems odd to me.

However, changing As visibility to public and it correctly gives

java.util.List<java.lang.String>

---

Question

I do not know if this is a bug or actually expected behavior. If it is expected, what is the reasoning behind it?

I am using OpenJDK 15 from AdoptOpenJDK:

// javac
javac 15.0.1

// java
openjdk version "15.0.1" 2020-10-20
OpenJDK Runtime Environment AdoptOpenJDK (build 15.0.1+9)
OpenJDK 64-Bit Server VM AdoptOpenJDK (build 15.0.1+9, mixed mode, sharing)

A friend could also reproduce it in:

• JDK Zulu 8
• AdoptOpenJDK 10, 11, 14

---

Findings

I experimented a lot and figured out that the only visibility-combination between A and B that triggers this issue is when B is public and A is not public. Any other combination and it works as expected again. So only

• B public, A protected
• B public, A package-visible
• B public, A private

show weird behavior.

I tried moving the code around in different files, putting it into different packages, adding or removing static here and there and nothing changed.

I also checked the source code of the method, which is

public Type getGenericReturnType() {
  if (getGenericSignature() != null) {
    return getGenericInfo().getReturnType();
  } else { return getReturnType();}
}

where getGenericSignature() relies on a String signature that is set during construction of the Method instance. It appears that it is null for some reason in above situation.

---

Update 1

I was just checking out the bytecode of the classes and found this in B.class:

  public Test$B();
    descriptor: ()V
    flags: (0x0001) ACC_PUBLIC
    Code:
      stack=1, locals=1, args_size=1
         0: aload_0
         1: invokespecial #1                  // Method Test$A."<init>":()V
         4: return
      LineNumberTable:
        line 16: 0

  public java.util.List foo();
    descriptor: ()Ljava/util/List;
    flags: (0x1041) ACC_PUBLIC, ACC_BRIDGE, ACC_SYNTHETIC
    Code:
      stack=1, locals=1, args_size=1
         0: aload_0
         1: invokespecial #7                  // Method Test$A.foo:()Ljava/util/List;
         4: areturn
      LineNumberTable:
        line 16: 0

To me this looks like B, for some reason, created another method foo() that simply forwards the method call to As foo() and hence has no generic type information.

Moreover, when calling B.getDeclaredMethods() it actually returns a method, namely

public java.util.List Test$B.foo()

Even though this method is supposed to exclude inherited methods (from the documentation):

Returns an array containing Method objects reflecting all the declared methods of the class or interface represented by this Class object, including public, protected, default (package) access, and private methods, but excluding inherited methods.

Which would now make sense if B really created a wrapper-method.

However, why is it creating such a method? Is there maybe a JLS section that explains this behavior?

Mucin

What happens when you use B.class.getDeclaredMethods()? getMethods() only picks up public members, maybe that has some impact. (Only an assumption) – Florrie

@Florrie Only yields one method, foo. toGenericString() on it gives public java.util.List Test$B.foo(). I am surprised by this, as this method is supposed to exclude inherited methods (from doc: but excluding inherited methods.). – Mucin

@Florrie Weird, the bytecode for B appears to list an additional foo() method (without generics) that just wraps As true foo() method (updated the question). – Mucin

I wanted to mention the same, I've found this answer which mentions that something funky happens with subclasses, but doesn't go into more detail... – Florrie

I think what happens is that the compiler generates a synthetic bridge method in B during type erasure of A#foo and that reflection is picking that up. That would explain the observations but I do not really know why visibility plays a role in that process. See docs.oracle.com/javase/tutorial/java/generics/… – Mucin

Still doesn't really answer the question why the bridge method is generated in the subclass. Because it doesn't override foo()... but I think you're on the right track – Florrie

@Florrie here you go – Jurgen

Answer 1

Let's take it slow here. First of all, here is why a bridge method is generated to begin with. Even if you drop generics, there will still be a bridge method. That is, this code:

static class A {
    public String foo() { return null; }
}

public static class B extends A {}

will still generate a foo method with ACC_PUBLIC, ACC_BRIDGE, ACC_SYNTHETIC. You can read the bug description and understand why that is needed.

On the other hand if you make A public, such a method will not be generated, the reason should be obvious, considering the previous bug explanation (I hope). So the idea is that if you have a non-public class, javac will generate a bridge method for the scenario above.

---

Now, if you add generics into that mix of synthetic methods, things start to shed some light. For example, you have this:

interface WithGeneric<T> {
    public WithGeneric<T> self(T s);
}

public class Impl implements WithGeneric<String> {

    @Override
    public WithGeneric<String> self(String s) {
        return null;
    }
}

There will be a bridge method generated too in Impl.class, but its declaration is going to be the erasure of the interface. In other words there will be two methods in Impl.class:

public WithGeneric<String> self(String) {...}

public WithGeneric self(Object) {...}

---

If you glue these two things:

• in case of non-public classes a bridge method will be created (so that reflection would work)

• in case of generics an erased bridge method will be created (so that erased calls would work)

things will make (somehow) sense.

Jurgen

That makes totally sense, thanks. Especially for the bug link, very insightful. Do you know why javac does not add the type signature to the bridge if it can? In my situation, I think, it could. I get that there are situations in which it can not keep the signature (as you have shown) but javac seems to not bother for simpler cases as well. If I manually add the signature in the bytecode it works again and seems to be accepted by the JVM. So Im just wondering why javac doesnt try to give the information in "safe cases" but maybe Im overlooking something. – Mucin

@Mucin I was thinking of exactly the same and you are right. I guess this is an edge case that javac devs did not find suitable handling. You could file a ticket and find out, may be this is indeed a bug. – Jurgen

I wonder whether this bugfix is still necessary with JEP 181 giving nest-based access control in Java 11. – Mucin

It is. Method.invoke checks if you can access the owner of the method - and as the owner of the method is package-private, you can't. To prevent that, you get a synthetic bridge method whose owner is the public class. – Rustler

Answer 2

When you declare A public, B.class.getMethods()[0] is not referencing B; It is referencing A.foo(), where the method is declared and the type is obtained because of the existance of the signature.

---

Declaring A non-public forces B.class.getMethods()[0] reference B.foo().

As there is no declaration of the inherited method, the type can't be obtained from the call to getGenericReturnType due to type erasure being applied to generics.

In compile time:

List<String> foo() becomes List foo().

And that's all the information B could give to you regarding the method's return type, since the signature in B.foo() was just removed.

---

Both A and B hold the same return type for foo() : java.util.List (without parameter type)

This is A.foo()'s declared return type. The same than B.foo():

The difference is that A has a valid signature, so it will complete the result from the call to getGenericReturnType() by adding the parameter types.

In the case of B, it will only show what it knows: just the return type.

---

I suffered strong headaches trying to solve the visibility puzzle. For a complete explanation of why this happens, look up at Eugene's answer. Seriously, you can learn a lot from this type of guy.

Asparagine

While this makes somewhat sense, I do not get why B.getDeclaredMethods() returns foo() although its meant to exclude inherited methods. If I got you right, your answer basically says that B does not have its own foo() method, yet I can see one in the bytecode and B.getDeclaredMethods() returns one as well. Sounds inconsistent to me. Do you maybe have some JLS sections that clarify the rules in this situation? – Mucin

it has a foo() method, and just like A, the return type for both is List (without type). The difference is A holds the signature because it declares the method and parameters, while B gets a null value signature. So the only information it gets from B.foo() is that returns a List. The last method post on your question will indeed say signature==null for B.foo, so you get the basic return type obtained from getReturnType(), which is simply List – Asparagine

But in the case of A, the signature won't be null, so the info you get is "complete", as it will call getGenericInfo().getReturnType() instead. – Asparagine

Why does B have a foo() method (a synthetic bridge method)? Or do you mean it inherited it from A? Then, why does B.getDlecaredMethods() return foo() although its not allowed to show any inherited methods? And why does it suddenly work if you increase the visibility of A but still stick to B.getMethods()[0]. The visibility should not change the information available to B, should it? After all, Test does have access to A as well. – Mucin

I don't really get what you are trying to say. The thing is: A has a signature; B doesn't. A returns the return type (List) + the info from the signature. – Asparagine

B doesn't have a signature. So it returns what he knows: the return type. In both cases, the return type is List. Without parameter types. – Asparagine

B does not declare its own foo() method. It only inherits one from A. Hence, getDeclaredMethods() should not show any method, yet it shows foo(). And your explanation seems to miss the point why B.getMethods()[0] suddenly has type information if the visibility of A is changed (although everyone in the snippet has still access to A). I think what happens here is that the compiler generated a synthetic bridge method in B for foo() during type erasure and that reflection is picking that up. I do not really know why the visibility plays a role though. – Mucin

B extends from A, so it must show foo() as a method from B. It is a declared method, just like as wait(), equals(), toString(), notify(), and so on. You can check it by looking at the result of B.class.getMethods(). -- – Asparagine

B.class.getDeclaredMethods() explicitly excludes any inherited methods. It does not show equals, wait, ... It should not show foo unless there is a bug or the compiler actually put a sneaky foo clone into it. And looking at docs.oracle.com/javase/tutorial/java/generics/… I think that is exactly what happens. I dont have the full rules for this process yet though. – Mucin

You are totally right, my bad there regarding declared methods. – Asparagine

So the issue is now getting to know how foo() becomes a declared method on B, when it is inherited and shouldn't to. The visibility of A is making a strange logic in there, as the javadocs are somehow against what's happening when A is private. – Asparagine

I think what happens here is that the compiler generated a synthetic bridge method in B for foo() during type erasure and that reflection is picking that up. Now I got you, and this makes a lot of sense. Trying this on different versions of java could show what changed on the logic of reflection and why this behaviour doesn't match with the official docs... – Asparagine

@Mucin It's not only happening with A's visibility, but also B's. If B isn't public, it cant find any declaredMethod if A isnt public either. So B will show foo() as declared method if A is not public + B is public. headache. – Asparagine

@Asparagine here is a pill – Jurgen

@Mucin also, thanks for this question, it made me learn something new today. And sorry for not getting your point before, I was stubborn here until I got what you were trying to say here. Thanks!! – Asparagine