Why doesn't Array class expose its indexer directly?

Question

something to mention for answering:

1. Don't worry about variance, while the item in question is Array rather than T[].

2. A similar case for multi-dimension arrays is [here]

That is, N-dims to linear transform, is always possible. So this question especially caught my attention, since it already implemented IList for a linear indexer.

---

Question:

In my code, I have following declaration:

public static Array ToArray<T>(this T source); 

My code knows how to make souce presents an array(at runtime). And I'm trying to allow the consuming code to access its indexer directly. But without "as IList", it cannot not be done. To return object[] might require extra converting/casting, that's what I'm preventing to do. What I can do is:

public static IList ToArray<T>(this T source); 

But I would think that a method named ToArray returns an IList looked strange.

Thus, I'm confused with that:

In the declaration of Array, there is

object IList.this[int index];

So that we can

Array a;
a=Array.CreateInstance(typeof(char), 1);
(a as IList)[0]='a';

But we cannot

a[0]='a';

except if it was declared as

public object this[int index]; 

The only difference I can see is that it requires we use its indexer explicitly through the interface IList by which it was implemented, but why? Are there benefits? Or are there exposing issues?

Knut

Why would you want to use Array like this, instead of char[]? – Kilocycle

@KenKin: If type shouldn't be enforced, you can still use my code below but cast as object[] to gain use of the indexer. – Pubis

@Pubis No, you can't. You can't cast char[] to object[], covariant array conversions only for work arrays of reference types. – Kilocycle

@KenKin Could you explain why isn't some of your code “explicit type declared”? Wouldn't object[] or generic T[] work for you? – Kilocycle

@KenKin I don't know your code, so I have no idea what argument are you taking about, what does this have to do with IEnumerable<T> or how exactly would you use T[]. Maybe you could ask another question about that. – Kilocycle

@KenKin No, you should return the right type of array. If it's different from T and you aren't able to figure it out statically from T, then the user will have to specify it: TResult[] ToArray<TSource, TResult>(this TSource source). – Kilocycle

KenKin I vote to close and removing my guess answer why it was done that way. Unless Eric Lippert decides that your question is insanely interesting you have more or less no chance to dig inner reasons of "why". My guess is "no explicit demand for feature and works", but it is just that - guess. – Burnsed

@Alexei Levenkov: I've got no idea that you suppose it was responsible of particular person to answer. But thanks for this idea. – Knut

@AlexeiLevenkov It's more of a change than addition of feature. Array already does have that indexer, except it's an explicit interface implementation, so it can't be used directly. – Kilocycle

I was not present when Array was designed, so my educated guess is no better than yours. I recommend against using Array. What I don't understand is why the signature of your method is not T[] ToArray<T>(this T source). – Heighho

@KenKin Note that a 1-dimensional array is always an IList<T>, while an n-dimensional array is not. Having an indexer at Array gives all n-dimensional arrays the 1-dimensional indexer, which you can argue to be a bad design choice. The other way around, you can also argue that Array should have been designed 1-dimensional and MultiDimensionalArray (give it a name...) as n-dimensional; apparently the C# team decided otherwise. I can imagine that this idea originated from Pascal. – Briefcase

Answer 1

Array can't have an indexer because it needs to be able to represent an array with any number of dimensions. The indexer for a two dimensional array has a different signature than for a one dimensional array.

If an indexer was provided and used on an Array that represented a two dimensional array what should happen?

The solution that the language designers choose was to just not include an indexer at all.

If you know that your ToArray method will always return a one dimensional array then consider using:

public static T[] ToArray<T>(this T source); 

That will have an indexer.

If the elements in the array will not all be of type T then you can return an object[]:

public static object[] ToArray<T>(this T source); 

Unbridle

@KenKin So if the source is a char[] then what should it return an array of? Based on your other linked question you probably just want to use the existing LINQ ToArray method, and use Cast on any non-generic IEnumerable sequences that you have first. It's possible to present a multi-dimensional array as one dimension, but apparently the language team choose not to do so; perhaps because they felt that it wasn't expected behavior. You could of course create your own method that takes an object that wraps an N dimensional array and exposes a single dimensional indexer. – Unbridle

@KenKin Sounds like you need to fix your input first; such a method shouldn't need to exist. The caller of your method will have no effective means of knowing what the type of the returned array is, so they can't effectively use it. You should have two entirely seperate methdods, one to convert a sequence of items into an array (namely, LINQ's ToArray) and another to convert a single item into an array of that item. You can compose the two methods as needed to get the desired result from the point of view of a particular caller. – Unbridle

@KenKin Because they choose not to expose it, or they never considered it as an option to begin with. It really doesn't matter much. Unless one of those few people happen to show up here, you'll never know, you can only guess. I have given you one reasonable reason why someone might choose not to add it. If you only care about what the language designers were thinking, and not any reason why you might not want to include an indexer, then this question should be closed as this isn't the proper audience for that type of question. – Unbridle

Answer 2

a as IList is (basically) casting. So just cast it first:

char[] a = (char[])Array.CreateInstance(typeof(char), 1);
a[0] = 'a';

Edit: the reason is: because the interface for Array simply doesn't define an indexer. It uses SetValue(Object, Int32) and Object GetValue(Int32). Notice the ominous Object stuff in there. Array isn't type specific; it's built for the lowest common denominator: Object. It could have just as easily defined an indexer, but in practice you'd still have the un/boxing problem.

Pubis

Yeah, but the question is, why doesn't it work without casting? – Kilocycle

Answer 3

I think one reason why Array doesn't implement that indexer directly is because all the specific array types (like char[]) derive from Array.

What this means is that code like this would be legal:

char[] array = new char[10];
array[0] = new object();

Code like this shouldn't be legal, because it's not type-safe. The following is legal and throws an exception:

char[] array = new char[10];
array.SetValue(new object(), 0);

But SetValue() is not normally used, so this is not a big problem.

Kilocycle

Nice answer. But this implying (array as IList)[0]=new object(); should not be legal, however, it is. – Knut

I think it shouldn't be legal, because arrays shouldn't implement the non-generic IList. But it's there for backwards compatibility with .Net 1.0, which didn't have generics. And without generics, there is no way to make that code illegal, while supporting some list interface. – Kilocycle

Yeah, I do think the non-generic types like IList, IEnumerable or ArrayList exist only for compatibility and that they shouldn't be used in new code. – Kilocycle

Surely you mean array.SetValue(object, 0). – Macdougall

Answer 4

The problem with IList<T>'s methods in the Array class, including its indexer, is that their explicit implementations are added to Array objects of the class at run time:

Starting with the .NET Framework 2.0, the Array class implements the System.Collections.Generic.IList<T>, System.Collections.Generic.ICollection<T>, and System.Collections.Generic.IEnumerable<T> generic interfaces. The implementations are provided to arrays at run time, and therefore are not visible to the documentation build tools. As a result, the generic interfaces do not appear in the declaration syntax for the Array class, and there are no reference topics for interface members that are accessible only by casting an array to the generic interface type (explicit interface implementations).

When classes implement interfaces explicitly, accessing interface methods requires a cast:

A class that implements an interface can explicitly implement a member of that interface. When a member is explicitly implemented, it cannot be accessed through a class instance, but only through an instance of the interface.

The problem with providing a "regular" (as opposed to an "explicit") interface implementation is the fact that the Array class is not generic: without a type parameter, you cannot write

class Array : IList<T>

simply because T is undefined. The environment cannot slap an interface implementation onto the Array class until the type of the T parameter becomes known, which may happen only at run time:

// The type of [T] is char
Array a = Array.CreateInstance(typeof(char), 1);
// The type of [T] is int
Array b = Array.CreateInstance(typeof(int), 1);
// The type of [T] is string
Array c = Array.CreateInstance(typeof(string), 1);

At the same time, the static type of a, b, and c remains the same - it's System.Array. However, at run time a will be implementing IList<char>, b will be implementing IList<int>, and c - IList<string>. None of it is known at compile time, prevents the compiler from "seeing" the indexer and other methods of IList<T>.

Moreover, not all instances of Array implement IList<T> - only arrays with a single dimension do:

Array x = new int[5];
Console.WriteLine(x.GetType().GetInterface("IList`1") != null); // True
Array y = new int[5,5];
Console.WriteLine(y.GetType().GetInterface("IList`1") != null); // False

All of the above prevents the compiler from accessing IList<T> methods, including the indexer, without an explicit cast.

Sauerkraut

@KenKin (1) I am confused: which document are you talking about? (2) Array implements IEnumerable statically, i.e. the fact that Array implements IEnumerable is known to the compiler. IEnumerable x = Array.CreateInstance(typeof (int), 1) works without a cast; IEnumerable<int> x = Array.CreateInstance(typeof (int), 1) does not. (3) as is indeed a casting operator: expression as type is equivalent precisely to expression is type ? (type)expression : (type)null. – Sauerkraut

I believe you can figure the points I told in the comment out. And what I don't understand is statically. Would you describe that for me? – Knut

@KenKin "I believe you can figure the points I told in the comment out." If I could, I wouldn't ask you to clarify. By "statically" I mean "declared in the source code". If you examine the source of Array.cs (link) you'll see on line 52 that the class implements IEnumerable. The IList<T> is not there, because it is added only at run time, and only to selected subclasses of the abstract Array class. – Sauerkraut

he(dasblinkenlight) already said: "When classes implement interfaces explicitly, accessing interface methods requires a cast" – Ozellaozen

@dasblinkenlight While your answer is correct, you're not mentioning everything. A few important markers: (1) a.GetType() != b.GetType() so they have the same name but are really different types (you are comparing something similar to base classes), (2) generics also have this property where the type is unknown and (3) indexing arrays like int[] isn't really an indexer but actually an opcode - hence that cannot be generalized to Array's. (4) Last, the cast is required because you're working on a base object created through reflection and is no different from Activator.CreateInstance. – Briefcase

Answer 5

Short answer:

System.Array is a base class for N-D arrays (not only 1-D), that's why 1-D indexer (object this[i]{get;set;}) cannot be a base member.

Long answer:

If you let's say create 2-dimensional array and try to access it's IList indexer:

Array a;
a=Array.CreateInstance(typeof(char), 1,1);
(a as IList)[0]='a';

You will get not supported exception.

Good question would be:

Why System.Array implement IList and IEnumerable while most of its implementation will throw NotSupportedException for non 1-D array??

One more interesting thing to mention. Technically non of the arrays have class-indexer internally in classic meaning. Classic meaning of Indexer is a property "Item" + get(+set) method(s). If you go deep to reflection you will see that typeof(string[]) does not have indexer property and it only has 2 methods Get and Set - those method declared in string[] class (not in base class, unlike Array.SetValue, Array.GetValue) and they are used for compile-time indexing.

Cyclic

In fact, Array exposed GetEnumerator() with the defination public IEnumerator GetEnumerator();. Most of time when Array throws expcetion, is indeed ArgumentException, either ArgumentOutOfRangeException or ArgumentNullException. It only throws NotSupportedException at four places: Add, Insert, Remove, RemoveAt those are all implementing of IList, and the reason is array is a fixed size collection. – Knut

What about exception on IList indexer for ND array? Yes it expose enumerator which returns flattened items of the array, which is kind of not obvious – Cyclic

It is not obvious for consumer that array {{11,12,13}, {21,22,23}} will return enumerator: 11, 12, 13, 21, 22, 23. – Cyclic

Answer 6

Even if array's were all 1D, you'd still have a Covariance and Contravariance issue:

If the base class had a

public Object this[int index] { get; set; }

indexer property, then the concrete types indexer properties

public TValue this[int index] { get; set; }

would collide with that of the base type (since the parameter is of the setter is the same however the return value isn't).

Casting the base class into either a base interface or a generic interface like either IList or IList solves this, since the non-specific indexer can be implemented explicitly. This is the same with the

Add(Object value)

vs.

Add(TValue value)

methods.

The multi dimensional issue could, theoretically, be overcome by defining a conversion between 1D indexes and n-D indexes (e.g. [n] = [n / length(0), n % length(0)]) since n-D matrices are stored as one continuous buffer.

Rooke

Would you give a sample of the situation that you described? – Knut

@KenKin Can you be more specific? If it's the whole issue, try defining 2 classes that do not inherit from array, a base class and a generic class that inherits from it and add an indexer to both and try to compile. – Rooke

The generic indexer just hides the base indexer, no matter get or set. – Knut

@KenKin I assume you added the "new" keyword for this to work smoothly. – Rooke

I didn't. Either I add or not, new is applied. – Knut

Answer 7

Technically there are two types of arrays. Vector types or Matrix types. The runtime refers to Vector types as Sz_Array and they are the type you get when you declare a 1d array*. I have no clue why. Matrix types represent multidimensional arrays. Unfortunately they both inherit from Array and no other intermediary types.

Why doesn't Array class expose its indexer directly?

The reason why you can only access the indexer for 1d arrays when you have it as a T[] is because the indexer for 1d arrays is implemented in the runtime through IL opcodes.

e.g.

static T GetFirst<T>(T[] a)
{
   return a[0]:
}

Translates to the following il::

L_0000: ldarg.0 
L_0001: ldc.i4.0 
L_0002: ldelem.any !!T
L_0007: ret 

Where as the following C#

private static T GetFirst<T>(IList<T> a)
{
    return a[0];
}

translates to this IL.

L_0000: ldarg.0 
L_0001: ldc.i4.0 
L_0002: callvirt instance !0 [mscorlib]System.Collections.Generic.IList`1<!!T>::get_Item(int32)
L_0007: ret 

So we can see that one is using an opcode ldelem.any, and the other is callvirt a method.

The runtime injects IList<T>,IEnumerable<T> at runtime for arrays. The logic for them in the MSIL is located in the class SZArrayHelper

The runtime which provides the implementation, also creates two helper methods for every array generated to help languages that do not support indexers (if such a language exists) C# does not expose them but they are valid methods to call. They are:

T Get(Int32)
void Set(Int32, T)

These methods are also generated for matrix types arrays depending on the dimension and are used by C# when you call indexers.

However, unless you actually specify that your is a typed array you don't get the indexer. As Array does not have an indexer method. The opcodes can't be used because at compile time you need to know that the array in question is a vector type and the element type of the array.

But Array implements IList! That has an indexer can't I call it?

Yes it does, but the implementation for the IList methods are explicit implementations so they can only be called in C# when cast or when bound by a generic constraint. Probably, because for any non vector type of array it throws a not supported exception when you call any of its methods. Since its only conditionally supported the creators of the run-time probably want you to cast it for times when you know for a fact this is a 1d array, but I can't name the type right now.

Why does array implement IList if for any multidimensional arrays the implementation throws not supported?

This is probably a mistake that was there since 1.0. They can't fix it now as someone for whatever reason might be casting a multidimensional array to an IList. In 2.0 they added some runtime magic to add implementation to the vector type classes at runtime so that only 1d arrays implement IList<T> and IEnumerable<T>.

And we can interpolate your next question::

How can I get the indexers to show up without casting? Change the method signature to something like::

public static T[] ToArray<T>(this T source)

But you might say your ToArray does not return a T[], It returns something else what do I do? If you can specify the return type explicitly just do that. If its a reference kind you can always abuse array covariance and change the return type to object[] but then you are at the mercy of ArrayTypeMismatchException. This won't work if you are getting a valuetype back as that cast is illegal. At this point you can just return IList but then you are boxing the elements and you are still at the mercy of ArrayTypeMismatchException. Array is the base class for all array types for a reason it has helper methods to help you access the content like GetValue and SetValue and you'll note they have overloads that take arrays of indices so that you can access elements in Nd as well as 1d arrays. e.g.

IList myArray = ToArray(myValues);
// myArray is actually a string array.
myArray[0]='a';
// blows up here as you can't explicitly cast a char to a string.

So the short of it is you don't know the explicit type. And every inheritor of Array implementing IList, even when it doesn't make much sense, is an implementation detail that they can't change since it was there since 1.0.

• Technically this isn't a 100% true. You can create a matrix type array that only has 1 dimension. This eldritch abomination can be created in IL or you can create the type using the typeof(int).MakeArrayType(1) Notice how you now have a System.Int32[*] instead of System.Int32[]

Inalienable

Answer 8

From msdn:

The Array class is the base class for language implementations that support arrays. However, only the system and compilers can derive explicitly from the Array class. Users should employ the array constructs provided by the language.

If it provide you an indexer it contradicts with the original intention of Array class. You should use the compiler implementation.

Again from msdn:

Important: Starting with the .NET Framework 2.0, the Array class implements the System.Collections.Generic.IList, System.Collections.Generic.ICollection, and System.Collections.Generic.IEnumerable generic interfaces. The implementations are provided to arrays at run time, and therefore are not visible to the documentation build tools. As a result, the generic interfaces do not appear in the declaration syntax for the Array class, and there are no reference topics for interface members that are accessible only by casting an array to the generic interface type (explicit interface implementations). The key thing to be aware of when you cast an array to one of these interfaces is that members which add, insert, or remove elements throw NotSupportedException.

It is an afterthought, I presume.

Skillern

Answer 9

C# Specs "12.1.1 The System.Array type" says, "Note that System.Array is not itself an array-type"

Because it is not an array type.

And note that "6.1.6 Implicit reference conversions" says, "From a single-dimensional array type S[] to System.Collections.Generic.IList and its base interfaces, provided that there is an implicit identity or reference conversion from S to T"

C# Specs: http://www.microsoft.com/en-us/download/details.aspx?id=7029

About why the indexer access is so much of a mystery, check this other SO post: implicit vs explicit interface implementation

hope it helps.

Ozellaozen

This is why there is no indexer as a result of itself being an array, but it doesn't explain why there's not a user-defined indexer for the Array class. They certainly could have added one if they wanted to. – Unbridle

That is C# Array class, which is itself not an array-type -- just the absract class for the array types. – Ozellaozen