What are Reified Generics? How do they solve Type Erasure problems and why can't they be added without major changes?

Question

I've read Neal Gafter's blog on the subject and am still unclear on a number of points.

Why is it not possible to create implementations of the Collections API that preserve type information given the current state of Java, the JVM and existing collections API? Couldn't these replace the existing implementations in a future version of Java in a way where backwards compatibility is preserved?

As an example:

List<T> list = REIList<T>(T.Class);

Where REIList is something like this:

public REIList<T>() implements List {
  private Object o;
  private Class klass;

  public REIList(Object o) {
    this.o = o;
    klass = o.getClass();
  }
... the rest of the list implementation ...

And the methods use Object o and Class klass to get the type information.

Why would preserving generic class information require language changes rather than just a JVM implementation change?

What am I not understanding?

Courtier

Answer 1

The whole point is that reified generics have support in the compiler for preserving type information, whereas type erased generics don't. AFAIK, the whole point of having type erasure in the first place was to enable backwards compatibility (e.g. lower versioned JVMs could still understand generic classes).

You can explicitly add the type information in the implementation, as you have above, but that requires additional code every time the list is used, and is pretty messy in my opinion. Also, in this case, you still don't have runtime type checking for all of the list methods unless you add the checks yourself, however reified generics will ensure the runtime types.

Dead

It's source and object code written for older JVMs can run on new JVMs without any changes (source code will have problems with "enum" as identifier, but that isn't generics as such). – Dispersion

"lower versioned JVMs could still understand generic classes" <-- I think you will get an UnsupportedClassVersionError when you try to run a program compiled with a newer version of JDK in an older version of JRE/JVM. So the fact should be what Tom Hawtin and Richard Gomes have said which is the opposite. – Teddy

Answer 2

Contrary to beliefs of majority of Java developers, it is possible to keep compile-time type information and retrieve this information at runtime, despite in a very restricted way. In other words: Java does provide reified generics in a very restricted way.

Regarding type erasure

Notice that, at compile-time, the compiler has full type information available but this information is intentionally dropped in general when the binary code is generated, in a process known as type erasure. This is done this way due to compatibility issues: The intention of language designers was providing full source code compatibility and full binary code compatibility between versions of the platform. If it was implemented differently, you would have to recompile your legacy applications when you migrate to newer versions of the platform. The way it was done, all method signatures are preserved (source code compatibility) and you don't need to recompile anything (binary compatibility).

Regarding reified generics in Java

If you need to keep compile-time type information, you need to employ anonymous classes. The point is: in the very special case of anonymous classes, it is possible to retrieve full compile-time type information at runtime which, in other words means: reified generics.

I've written an article about this subject:

http://rgomes-info.blogspot.co.uk/2013/12/using-typetokens-to-retrieve-generic.html

In the article, I describe how our users reacted to the technique. In a nutshell, this is an obscure subject and the technique (or the pattern, if you prefer) looks extraneous to majority of Java developers.

Sample code

The article I've mentioned above has links to source code which exercises the idea.

Generality

This has nothing to do with reified generics. Yes, generics in class declarations like the types in generic parameters, types of fields, types in method signatures, type of superclass, type of enclosing class of inner classes (there is nothing special about anonymous classes) are stored in the bytecode because other classes that use this class and only has the .class file need to know these things to enforce generics. However, the discussion of reified vs. non-reified generics talks about the generic type of runtime objects, which is unrelated to what you're talking about above. – Tureen

@newacct: Yes, it does. Yes, inner classes are special from the point of view they preserve compile-time information which can be retrieved at runtime. By definition, when you can, at runtime, retrieve compile-time type information from generic classes... what do you have? – Generality

You can also equally retrieve the compile-time information about the superclass, superinterfaces, fields, and methods, etc. for ANY class. – Tureen

@newacct: Correct, but anonymous classes provide an additional feature: you can retrieve type information for their generic types at runtime. In other words: you do have reified generics for anonymous classes. See this test case: jquantlib.org/sites/jquantlib/xref-test/org/jquantlib/testsuite/… – Generality

No, you are simply retrieving the generic type parameter with which it extended its superclass. The anonymous class creation expression creates an instance of a class which is a subclass of the type you gave. final K k1 = new K<java.lang.Double>() {}; is exactly the same as the local class class Foo extends K<java.lang.Double> {}; final K k1 = new Foo(); – Tureen

@Tureen :: that link was down. Please consult this one: bazaar.launchpad.net/~frgomes/jquantlib/trunk/view/1362/… – Generality

@Tureen :: The point is: the solution I've presented is the closer you can get to reified generics. The code generated (the callee site) is still only one copy for all live types employed (at the caller site). If you tell me that it is not truly reified due to such reason... yeah... I agree. But the question is about retrieving runtime information as if you had reified generics, in the context of Neal Gafter's article. If a developer values true reified generics, with multiple copies at the callee site... oh well... better look for another programming language :) – Generality

That is not in the least what the question is about. It is about why would the reified implementation described by the OP require language changes rather than just JVM changes. Your answer is about neither. – Forget

The article didn't use the term reified even once, and how does it answer the question? It's a nice article by the way, in it's own concept – Antihistamine

Answer 3

IIRC (and based on the link), Java generics are just syntactic sugar for the existing technique of using an Object collection and casting back and forth. It's safer and simpler using the Java generics, since the compiler can do the checks for you to verify that you maintain compile-time type safety. Run time, however, is an entirely different problem.

.NET generics, on the other hand, actually create new types - a List<String> in C# is a different type than a List<Int>. In Java, under the covers, they are the same thing - a List<Object>. This means that if you have one of each, you can't look at them at run time and see what they were declared as - only what they are now.

The reified generics would change that, giving Java developers the same capabilities that exist now in .NET.

Agential

I put your List<*>s in backquotes so the types would show up. – Edelweiss

@David - thanks. As many times as I've done that for someone else, you think I'd remember... – Agential

You can mimic sort of reified generics with anonymous classes, i.e: type information is available at runtime in this circumstance. In spite it can be done, its applicability is very limited and the technique is very tricky. Reified Generics is a big miss; this is certainly true. – Generality

Answer 4

I'm no expert on the subject, but as I understand it the type information is lost at compile time. Unlike in C++, Java does not use a template system, type safety is achieved entirely through the compiler. At runtime, a List is actually a List, always.

So my take is that a change in the language specification is required due to the fact that the type information is not available to the JVM because it isn't there.

Garling

This is true in most cases except when you have anonymous classes. I mean: you can avoid type erasure when you employ anonymous classes. – Generality

To be a bit more specific about the subject: Java will not generate multiple copies of your code, like C++ does for templates. Java generates a single copy on the callee site, which employs Objects. So, despite that runtime type information is available in the case of anonymous classes, that's all you have: the information itself. The bytecode does not target the actual types you will be passing at runtime. – Generality

Answer 5

Simple explanation I can think of - they deliberately chose simplicity over code bloat, because of erasure after code is compiled there is only one implementation of a generic class and its subtypes.

C++ on the other hand create separate version of code for each implementation.

Mikimikihisa