Count the items from a IEnumerable<T> without iterating?

Question

private IEnumerable<string> Tables
{
    get
    {
        yield return "Foo";
        yield return "Bar";
    }
}

Let's say I want iterate on those and write something like processing #n of #m.

Is there a way I can find out the value of m without iterating before my main iteration?

I hope I made myself clear.

Malleolus

Answer 1

IEnumerable doesn't support this. This is by design. IEnumerable uses lazy evaluation to get the elements you ask for just before you need them.

If you want to know the number of items without iterating over them you can use ICollection<T>, it has a Count property.

Auriferous

I'd favor ICollection over IList if you don't need to access the list by indexer. – Mccubbin

I usually just grab List and IList out of habit. But especially if you want to implement them yourself ICollection is easier and also has the Count property. Thanks! – Auriferous

So how do I check the count when I have a given IEnumerable? – Dailey

@Shimmy You iterate and count the elements. Or you call Count() from the Linq namespace that does this for you. – Auriferous

Should simply replacing IEnumerable with IList be sufficient under normal circumstances? – Jetsam

@Helgi Because IEnumerable is evaluated lazily you can use it for things IList can't be used. You can build a function that returns an IEnumerable that enumerates all the decimals of Pi for example. As long as you never try to foreach over the complete result it should just work. You can't make an IList containing Pi. But that's all pretty academic. For most normal uses I completely agree. If you need Count you need IList. :-) – Auriferous

Makes sense, thanks. :) I converted my IEnumerables to IList and it was no problem at all! – Jetsam

So what if you don't know what type the incoming list will be (dictionary, list, etc.). You want your method to be flexible and allow the caller to specify that type. But you still need to iterate that list or dictionary incoming? I guess then you'd have to check what type was passed inside your method to see if it is a whatever then do whatever – Zaxis

what if you need a dictionary? – Zaxis

@CoffeeAddict: There's nothing wrong with using an IEnumerable, I usually use this type in method signatures to be more flexible just like you said. But if you really need to get the number of elements in the collection without enumerating it then using IList (or even better ICollection as Muchael Meadows suggested) is the way to go. If I'm not mistaken IDictionary inherits from IList too as a list of key value pairs. – Auriferous

@Mendelt: An advantage of the non-generic ICollection is that it can be used without knowing the actual item type associated with a generic collection. For example, suppose a routine which accepts IEnumerable<Animal> is given an instance of DogHouse, which happens to implement IList<Dog> and ICollection. It's much easier to check whether an IEnumerable<Animal> implements ICollection than to find a generic implementation of IList<T> that it implements. – Stannite

@Zaxis - to handle different types of collections EFFICIENTLY, you sometimes need to write several versions of a method. This violates DRY principle, but without some form of macro-preprocessing, it is a trade-off that is sometimes unavoidable in C#. If count helps performance, then two variations: abc<T>(IEnumerable<T>) and the faster abc<T>(IEnumerable<T>, int count). Plus some convenience methods: abc<T>(ICollection<T> c) { abc(c, c.Count);, abc<TK,TV>(IDictionary<TK,TV> d) { abc(d.Values, d.Count); } List implements ICollection, so is handled by that. – Outsider

Answer 2

The System.Linq.Enumerable.Count extension method on IEnumerable<T> has the following implementation:

ICollection<T> c = source as ICollection<TSource>;
if (c != null)
    return c.Count;

int result = 0;
using (IEnumerator<T> enumerator = source.GetEnumerator())
{
    while (enumerator.MoveNext())
        result++;
}
return result;

So it tries to cast to ICollection<T>, which has a Count property, and uses that if possible. Otherwise it iterates.

So your best bet is to use the Count() extension method on your IEnumerable<T> object, as you will get the best performance possible that way.

Best

Very interesting the thing that it tries to cast to ICollection<T> first. – Dashed

@OscarMederos Most of the extension methods in Enumerable have optimizations for sequences of the various types where they will use the cheaper way if they can. – Ody

The mentioned extension is available since .Net 3.5 and documented in MSDN. – Herzig

I think you don't need the generic type T on IEnumerator<T> enumerator = source.GetEnumerator(), simply you can do this: IEnumerator enumerator = source.GetEnumerator() and should work! – Wanting

@Wanting - it's slightly more complicated than that. IEnumerable<T> inherits IDisposable which allows the using statement to dispose it automatically. IEnumerable does not. So if you call GetEnumerator either way, you should finish off with var d = e as IDisposable; if (d != null) d.Dispose(); – Best

@Wanting - also depends what you want to do with the result. Usually, you WANT the type <T> to be explicit. Otherwise, you have to cast from the objects of GetEnumerator to the expected type, as you use them. If you make a mistake, e.g. pass around a collection whose elements are a different type than you intended, its a runtime exception instead of a compile-time error. – Outsider

Its a shame that .NET doesn't have an intermediate interface, between IEnumerable and ICollection, for IEnumerables that have a pre-known count (but don't need the other features of ICollection). Nor is there any way to know, given an arbitrary IEnumerable, and a need to know "Count" early in the algorithm, whether it would be cheaper to call IEnumerable.Count, thereby iterating collection a second time, or to collect into a list once, so that you have a count, then use that list. Worse, there may be "side-effects" of iterating twice. On the other hand, the enumeration might have MANY elements – Outsider

@DanielEarwicker Or an alternative syntax: (enumerator as IDisposable)?.Dispose() – Bergerac

@DanielEarwicker it's the IEnumerator<T> that implements the IDisposable interface, not the IEnumerable<T>. – Suffragist

"So your best bet is to use the Count() extension method on your IEnumerable<T> object, as you will get the best performance possible that way." <--- This conclusion is true ONLY WHEN the type you're working with implements ICollection<T>. If the type only implements IEnumerable<T>, full iteration will occur. A big lesson from this answer is that when you implement IEnumerable<T> on a custom Type, don't stop there. Also implement IReadOnlyCollection<T> and then implement Count {get;} so that your users will get optimal performance when they access your type. – Torsion

Answer 3

Just adding extra some info:

The Count() extension doesn't always iterate. Consider Linq to Sql, where the count goes to the database, but instead of bringing back all the rows, it issues the Sql Count() command and returns that result instead.

Additionally, the compiler (or runtime) is smart enough that it will call the objects Count() method if it has one. So it's not as other responders say, being completely ignorant and always iterating in order to count elements.

In many cases where the programmer is just checking if( enumerable.Count != 0 ) using the Any() extension method, as in if( enumerable.Any() ) is far more efficient with linq's lazy evaluation as it can short-circuit once it can determine there are any elements. It's also more readable

Multifold

In regards to collections and arrays. If you happen to use a collection then use the .Count property as it always knows it's size. When querying collection.Count there is no additional computation, it simply returns the already known count. Same for Array.length as far as I know. However, .Any() gets the enumerator of the source using using (IEnumerator<TSource> enumerator = source.GetEnumerator()) and returns true if it can do enumerator.MoveNext(). For collections: if(collection.Count > 0), arrays: if(array.length > 0) and on enumerables do if(collection.Any()). – Parshall

The first point is not strictly true... LINQ to SQL uses this extension method which is not the same as this one. If you use the second, the count is performed in memory, not as a SQL function – Laclos

@AlexFoxGill is correct. If you explicitly cast your IQueryably<T> to a IEnumerable<T> it will not issue a sql count. When I wrote this, Linq to Sql was new; I think I was just pushing to use Any() because for enumerables, collections and sql it was much more efficient and readable (in general). Thanks for improving the answer. – Multifold

Answer 4

Alternatively you can do the following:

Tables.ToList<string>().Count;

Nonviolence

Answer 5

A friend of mine has a series of blog posts that provide an illustration for why you can't do this. He creates function that return an IEnumerable where each iteration returns the next prime number, all the way to ulong.MaxValue, and the next item isn't calculated until you ask for it. Quick, pop question: how many items are returned?

Here are the posts, but they're kind of long:

1. Beyond Loops (provides an initial EnumerableUtility class used in the other posts)
2. Applications of Iterate (Initial implementation)
3. Crazy Extention Methods: ToLazyList (Performance optimizations)

Defaulter

I really really wish MS had defined a way to ask enumerables to describe what they can about themselves (with "not knowing anything" being a valid answer). No enumerable should have any difficulty answering questions like "Do you know yourself to be finite", "Do you know yourself to be finite with less than N elements", and "Do you know yourself to be infinite", since any enumerable could legitimately (if unhelpfully) answer "no" to all of them. If there were a standard means for asking such questions, it would be much safer for enumerators to return endless sequences... – Stannite

...(since they could say that they do so), and for code to assume that enumerators which don't claim to return endless sequences are likely to be bounded. Note that including a means of asking such questions (to minimize boilerplate, probably have a property return an EnumerableFeatures object) wouldn't require enumerators to do anything difficult, but being able to ask such questions (along with some others like "Can you promise to always return the same sequence of items", "Can you safely be exposed to code that shouldn't alter your underlying collection", etc.) would be very useful. – Stannite

That would be pretty cool, but I'm now sure how well that would mesh with iterator blocks. You'd need some kind of special "yield options" or something. Or maybe use attributes to decorate the iterator method. – Defaulter

Iterator blocks could, in the absence of any other special declarations, simply report that they don't know anything about the returned sequence, though if IEnumerator or an MS-endorsed successor (which could be returned by GetEnumerator implementations that knew of its existence) were to support additional information, C# would likely get a set yield options statement or something similar to support it. If properly designed, an IEnhancedEnumerator could make things like LINQ much more usable by eliminating a lot of "defensive" ToArray or ToList calls, especially... – Stannite

...in cases where things like Enumerable.Concat are used to combine a large collection which knows a lot about itself with a small one that doesn't. – Stannite

Answer 6

IEnumerable cannot count without iterating.

Under "normal" circumstances, it would be possible for classes implementing IEnumerable or IEnumerable<T>, such as List<T>, to implement the Count method by returning the List<T>.Count property. However, the Count method is not actually a method defined on the IEnumerable<T> or IEnumerable interface. (The only one that is, in fact, is GetEnumerator.) And this means that a class-specific implementation cannot be provided for it.

Rather, Count it is an extension method, defined on the static class Enumerable. This means it can be called on any instance of an IEnumerable<T> derived class, regardless of that class's implementation. But it also means it is implemented in a single place, external to any of those classes. Which of course means that it must be implemented in a way that is completely independent of these class' internals. The only such way to do counting is via iteration.

Mayenne

that's a good point about not being able to count unless you iterate. The count functionality is tied to the classes that implement IEnumerable....thus you have to check what type IEnumerable is incoming (check by casting) and then you know that List<> and Dictionary<> have certain ways to count and then use those only after you know the type. I found this thread very useful personally so thanks for your reply as well Chris. – Zaxis

According to Daniel's answer this answer is not strictly true: the implementation DOES check if the object implements "ICollection", which has a "Count" field. If so, it uses it. (Whether it was that smart in '08, I don't know.) – Outsider

Answer 7

No, not in general. One point in using enumerables is that the actual set of objects in the enumeration is not known (in advance, or even at all).

Athey

The important point you brought up is that even when you get that IEnumerable object, you have to see if you can cast it to figure out what type it is. That's a very important point to those trying to use more IEnumerable like myself in my code. – Zaxis

Answer 8

You can use System.Linq.

using System;
using System.Collections.Generic;
using System.Linq;

public class Test
{
    private IEnumerable<string> Tables
    {
        get {
             yield return "Foo";
             yield return "Bar";
         }
    }

    static void Main()
    {
        var x = new Test();
        Console.WriteLine(x.Tables.Count());
    }
}

You'll get the result '2'.

Justice

This does not work for the non-generic variant IEnumerable (without type specifier) – Jenisejenkel

The implementation of .Count is enumerating all the items if you have an IEnumerable. (different for ICollection). The OP question is clearly "without iterating" – Cavour

Answer 9

I think the easiest way to do this

Enumerable.Count<TSource>(IEnumerable<TSource> source)

Reference: system.linq.enumerable

Little

The question is "Count the items from a IEnumerable<T> without iterating?" How does this answer that? – Fenner

Did the author mean without iterating in code they wrote or without iterating anywhere at all. I think this is a valid answer. – Mydriatic

Answer 10

I used such way inside a method to check the passed in IEnumberable content

if( iEnum.Cast<Object>().Count() > 0) 
{

}

Inside a method like this:

GetDataTable(IEnumberable iEnum)
{  
    if (iEnum != null && iEnum.Cast<Object>().Count() > 0) //--- proceed further

}

Changchangaris

Why do that? "Count" might be expensive, so given an IEnumerable, it is cheaper to initialize whatever outputs you need to appropriate defaults, then start iterating over "iEnum". Choose defaults such that an empty "iEnum", which never executes the loop, ends up with a valid result. Sometimes, this means adding a boolean flag to know whether the loop was executed. Granted that is clumsy, but relying on "Count" seems unwise. If need this flag, code looks like: bool hasContents = false; if (iEnum != null) foreach (object ob in iEnum) { hasContents = true; ... your code per ob ... }. – Outsider

... it is also easy to add special code that should be done only the first time, or only on iterations other than the first time: `... { if (!hasContents) { hasContents = true; ..one time code..; } else { ..code for all but first time..} ...}" Admittedly, this is clumsier than your simple approach, where one-time code would be inside your if, before the loop, but if the cost of ".Count()" might be a concern, then this is the way to go. – Outsider

Answer 11

Going beyond your immediate question (which has been thoroughly answered in the negative), if you're looking to report progress whilst processing an enumerable, you might want to look at my blog post Reporting Progress During Linq Queries.

It lets you do this:

BackgroundWorker worker = new BackgroundWorker();
worker.WorkerReportsProgress = true;
worker.DoWork += (sender, e) =>
      {
          // pretend we have a collection of 
          // items to process
          var items = 1.To(1000);
          items
              .WithProgressReporting(progress => worker.ReportProgress(progress))
              .ForEach(item => Thread.Sleep(10)); // simulate some real work
      };

Embolectomy

Answer 12

There is a new method in LINQ for .NET 6 Watch https://www.youtube.com/watch?v=sIXKpyhxHR8

Tables.TryGetNonEnumeratedCount(out var count)

Folderol

Answer 13

Simplifying all answer.

IEnumerable has not Count function or property. To get this, you can store count variable (with foreach, for example) or solve using Linq to get count.

If you have:

IEnumerable<> products

Then:

Declare: "using System.Linq;"

To Count:

products.ToList().Count

Crackup

Answer 14

It depends on which version of .Net and implementation of your IEnumerable object. Microsoft has fixed the IEnumerable.Count method to check for the implementation, and uses the ICollection.Count or ICollection< TSource >.Count, see details here https://connect.microsoft.com/VisualStudio/feedback/details/454130

And below is the MSIL from Ildasm for System.Core, in which the System.Linq resides.

.method public hidebysig static int32  Count<TSource>(class 

[mscorlib]System.Collections.Generic.IEnumerable`1<!!TSource> source) cil managed
{
  .custom instance void System.Runtime.CompilerServices.ExtensionAttribute::.ctor() = ( 01 00 00 00 ) 
  // Code size       85 (0x55)
  .maxstack  2
  .locals init (class [mscorlib]System.Collections.Generic.ICollection`1<!!TSource> V_0,
           class [mscorlib]System.Collections.ICollection V_1,
           int32 V_2,
           class [mscorlib]System.Collections.Generic.IEnumerator`1<!!TSource> V_3)
  IL_0000:  ldarg.0
  IL_0001:  brtrue.s   IL_000e
  IL_0003:  ldstr      "source"
  IL_0008:  call       class [mscorlib]System.Exception System.Linq.Error::ArgumentNull(string)
  IL_000d:  throw
  IL_000e:  ldarg.0
  IL_000f:  isinst     class [mscorlib]System.Collections.Generic.ICollection`1<!!TSource>
  IL_0014:  stloc.0
  IL_0015:  ldloc.0
  IL_0016:  brfalse.s  IL_001f
  IL_0018:  ldloc.0
  IL_0019:  callvirt   instance int32 class [mscorlib]System.Collections.Generic.ICollection`1<!!TSource>::get_Count()
  IL_001e:  ret
  IL_001f:  ldarg.0
  IL_0020:  isinst     [mscorlib]System.Collections.ICollection
  IL_0025:  stloc.1
  IL_0026:  ldloc.1
  IL_0027:  brfalse.s  IL_0030
  IL_0029:  ldloc.1
  IL_002a:  callvirt   instance int32 [mscorlib]System.Collections.ICollection::get_Count()
  IL_002f:  ret
  IL_0030:  ldc.i4.0
  IL_0031:  stloc.2
  IL_0032:  ldarg.0
  IL_0033:  callvirt   instance class [mscorlib]System.Collections.Generic.IEnumerator`1<!0> class [mscorlib]System.Collections.Generic.IEnumerable`1<!!TSource>::GetEnumerator()
  IL_0038:  stloc.3
  .try
  {
    IL_0039:  br.s       IL_003f
    IL_003b:  ldloc.2
    IL_003c:  ldc.i4.1
    IL_003d:  add.ovf
    IL_003e:  stloc.2
    IL_003f:  ldloc.3
    IL_0040:  callvirt   instance bool [mscorlib]System.Collections.IEnumerator::MoveNext()
    IL_0045:  brtrue.s   IL_003b
    IL_0047:  leave.s    IL_0053
  }  // end .try
  finally
  {
    IL_0049:  ldloc.3
    IL_004a:  brfalse.s  IL_0052
    IL_004c:  ldloc.3
    IL_004d:  callvirt   instance void [mscorlib]System.IDisposable::Dispose()
    IL_0052:  endfinally
  }  // end handler
  IL_0053:  ldloc.2
  IL_0054:  ret
} // end of method Enumerable::Count

Missive

Answer 15

Result of the IEnumerable.Count() function may be wrong. This is a very simple sample to test:

using System;
using System.Collections.Generic;
using System.Linq;
using System.Collections;

namespace Test
{
  class Program
  {
    static void Main(string[] args)
    {
      var test = new[] { 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17 };
      var result = test.Split(7);
      int cnt = 0;

      foreach (IEnumerable<int> chunk in result)
      {
        cnt = chunk.Count();
        Console.WriteLine(cnt);
      }
      cnt = result.Count();
      Console.WriteLine(cnt);
      Console.ReadLine();
    }
  }

  static class LinqExt
  {
    public static IEnumerable<IEnumerable<T>> Split<T>(this IEnumerable<T> source, int chunkLength)
    {
      if (chunkLength <= 0)
        throw new ArgumentOutOfRangeException("chunkLength", "chunkLength must be greater than 0");

      IEnumerable<T> result = null;
      using (IEnumerator<T> enumerator = source.GetEnumerator())
      {
        while (enumerator.MoveNext())
        {
          result = GetChunk(enumerator, chunkLength);
          yield return result;
        }
      }
    }

    static IEnumerable<T> GetChunk<T>(IEnumerator<T> source, int chunkLength)
    {
      int x = chunkLength;
      do
        yield return source.Current;
      while (--x > 0 && source.MoveNext());
    }
  }
}

Result must be (7,7,3,3) but actual result is (7,7,3,17)

Molar

Answer 16

Here is a great discussion about lazy evaluation and deferred execution. Basically you have to materialize the list to get that value.

Raseta

Answer 17

The best way I found is count by converting it to a list.

IEnumerable<T> enumList = ReturnFromSomeFunction();

int count = new List<T>(enumList).Count;

Exaction

Answer 18

This works for me.

Assembly mscorlib, Version=4.0.0.0  
public interface IEnumerable<out T> : IEnumerable

int count = myIEnum.AsEnumerable().Count();

Urethritis

Answer 19

I would suggest calling ToList. Yes you are doing the enumeration early, but you still have access to your list of items.

Allembracing

Answer 20

It may not yield the best performance, but you can use LINQ to count the elements in an IEnumerable:

public int GetEnumerableCount(IEnumerable Enumerable)
{
    return (from object Item in Enumerable
            select Item).Count();
}

Landowska

How does the result differ from simply doing "`return Enumerable.Count();"? – Outsider

Good question, or is it actually an answer to this Stackoverflow question. – Landowska

Answer 21

In a debug console, you can use

?System.Linq.Enumerable.Count(myIEnumerable)
12

Selfdenial

Answer 22

I use IEnum<string>.ToArray<string>().Length and it works fine.

Lambrecht

This should work fine. IEnumerator<Object>.ToArray<Object>.Length – Vincentvincenta

Why do you do this, rather than the more concise and quicker-performing solution already given in Daniel's highly-upvoted answer written three years earlier than yours, "IEnum<string>.Count();"? – Outsider

You are right. Somehow I overlooked Daniel's answer, maybe because he quoted from the implementation and I thought that he implemented an extension method and I was looking for a solution with less code. – Tench

What is the most accepted way of handling my bad answer? Should I delete it? – Tench

Answer 23

I use such code, if I have list of strings:

((IList<string>)Table).Count

Plutus

Let's hope that the underlying connection can be cast to an IList<string> then... (and no, I'm not talking about the string part, just that not all IEnumerable<T> are backed by a type that implements IList<T>) – Sheffy