Why is ClassManifest needed with Array but not List?

Question

Define the following code:

import scala.collection.JavaConversions._  
val iter:java.util.Iterator[Any] = Array[Any](1, 2, 3).iterator
def func(a:Any):String = a.toString

def test[T:ClassManifest](iter:java.util.Iterator[Any], func:Any=>T):Array[T] =  
  iter.map(i=>func(i)).toArray

def testFunc = test(iter, func)

Here, I need to use ClassManifest for it to compile correctly, otherwise I get the error:

scala> def test[T](iter:java.util.Iterator[Any], func:Any=>T):Array[T] = 
     |   iter.map(i=>func(i)).toArray         

<console>:11: error: could not find implicit value for evidence parameter of 
type ClassManifest[T]
     iter.map(i=>func(i)).toArray
                          ^

On the other hand, the alternate code below using List does not require this and compiles fine.

import scala.collection.JavaConversions._  
val iter:java.util.Iterator[Any] = Array[Any](1, 2, 3).iterator
def func(a:Any):String = a.toString 

def test1[T](iter:java.util.Iterator[Any], func:Any=>T):List[T] = 
  iter.map(i=>func(i)).toList   


def testFunc1 = test1(iter, func).toArray

Note that the final output of testFunc and testFunc1 are identical.

How come the List version does not require a ClassManifest?

Unrestraint

Answer 1

Arrays in Java are not type-erased, and, in particular, an Array[Int] is different than an Array[Object] at the JVM level.

For any other class, type parameters are erased to Object, so List[Int] and List[Object] have the same representation at the JVM level.

Frequently

Yours and Angel's answer combined together form the correct answer :) – Unrestraint

Answer 2

Method toArray creates new array with elements of type T. And according to documentation:

So depending on the actual type parameter for T, this could be an Array[Int], or an Array[Boolean], or an array of some of the other primitive types in Java, or an array of some reference type. But these types have all different runtime representations, so how is the Scala runtime going to pick the correct one? In fact, it can't do that based on the information it is given, because the actual type that corresponds to the type parameter T is erased at runtime.

Gate

Answer 3

The short answer is because that's how the methods are defined in the API:

def toArray [B >: A] (implicit arg0: ClassManifest[B]) : Array[B]
def toList : List[A]

If you leave off the :ClassManifest in def test[T:ClassManifest] in your code, then all the compiler knows is that it has some unknown type T and therefore the compiler has no way of finding a ClassManifest for that type.

Why does the code need a ClassManifest? If you look at the source for toArray you'll see:

val result = new Array[B](size)

This constructor of Array requires a ClassManifest. See Easy Angel's answer for the documentation of this. Here's an example demonstrating it in the REPL:

scala> def createArray[T] = new Array[T](10)
<console>:5: error: cannot find class manifest for element type T
       def createArray[T] = new Array[T](10)

So basically, you have to write T: ClassManifest because Scala needs a ClassManifest in order to create a new array.

Celebes

Answer 4

Scala uses JVM native arrays as implementation for Array. Such can only be created with known type.

Since Sun decided to create legacy artifacts, generic types are erased and no longer present in byte code. That means that at runtime, Array[T] does no longer know the value of T and therefore, the VM can not create the array. There are some more complicated pitfalls you get when you try being compatible to Java 1.4 and introducing generics on array (I don't get the whole depth either), but bottom line is: JVM/Java's arrays are not generic; Scala helps us with the generic abstraction.

The class manifest you mention is, consequently, essentially a hack. It ensures statically that type information is available when an array is to be created by demanding this implicit parameter. In practice, any method/function that creates an array of a type parameter has to demand an implicit manifest, and all its callees and so on up to the point were the type is known (statically).

Yawning

"presumably because the objects are actually put on the heap sequentially" No, they aren't. Arrays of objects store references sequentially, not objects themselves. (This doesn't apply to arrays of primitives, of course.) – Adigun