Appending/concatenating two IEnumerable sequences

Question

I have two sets of datarows. They are each IEnumerable. I want to append/concatenate these two lists into one list. I'm sure this is doable. I don't want to do a for loop and noticed that there is a Union method and a Join method on the two Lists. Any ideas?

Opiumism

Are the sets of datarows of the same type? Different types? – Napoleonnapoleonic

@Obed....they are of the same type – Opiumism

Answer 1

Assuming your objects are of the same type, you can use either Union or Concat. Note that, like the SQL UNION keyword, the Union operation will ensure that duplicates are eliminated, whereas Concat (like UNION ALL) will simply add the second list to the end of the first.

IEnumerable<T> first = ...;
IEnumerable<T> second = ...;

IEnumerable<T> combined = first.Concat(second);

or

IEnumerable<T> combined = first.Union(second);

If they are of different types, then you'll have to Select them into something common. For example:

IEnumerable<TOne> first = ...;
IEnumerable<TTwo> second = ...;

IEnumerable<T> combined = first.Select(f => ConvertToT(f)).Concat(
                          second.Select(s => ConvertToT(s)));

Where ConvertToT(TOne f) and ConvertToT(TTwo s) represent an operation that somehow converts an instance of TOne (and TTwo, respectively) into an instance of T.

Microphyte

@Adam....I spoke too soon. I have done the Union I am getting duplicate records. Do Ineed to initialize a key property of some kind? – Opiumism

@MikeTWebb: Union has an overload that allows you to specify an IEqualityComparer. – Microphyte

Answer 2

I just encountered a similar situation where I need to concatenate multiple sequences.

Naturally searched for existing solutions on Google/StackOverflow, however did not find anything the did not evaluate the enumerable, e.g. convert to array then use Array.Copy() etc., so I wrote an extension and static utiltiy method called ConcatMultiple.

Hope this helps anyone that needs to do the same.

/// <summary>
/// Concatenates multiple sequences
/// </summary>
/// <typeparam name="TSource">The type of the elements of the input sequences.</typeparam>
/// <param name="first">The first sequence to concatenate.</param>
/// <param name="source">The other sequences to concatenate.</param>
/// <returns></returns>
public static IEnumerable<TSource> ConcatMultiple<TSource>(this IEnumerable<TSource> first, params IEnumerable<TSource>[] source)
{
    if (first == null)
        throw new ArgumentNullException("first");

    if (source.Any(x => (x == null)))
        throw new ArgumentNullException("source");

    return ConcatIterator<TSource>(source);
}

private static IEnumerable<TSource> ConcatIterator<TSource>(IEnumerable<TSource> first, params IEnumerable<TSource>[] source)
{
    foreach (var iteratorVariable in first)
        yield return iteratorVariable;

    foreach (var enumerable in source)
    {
        foreach (var iteratorVariable in enumerable)
            yield return iteratorVariable;
    }
}

/// <summary>
/// Concatenates multiple sequences
/// </summary>
/// <typeparam name="TSource">The type of the elements of the input sequences.</typeparam>        
/// <param name="source">The sequences to concatenate.</param>
/// <returns></returns>
public static IEnumerable<TSource> ConcatMultiple<TSource>(params IEnumerable<TSource>[] source)
{
    if (source.Any(x => (x == null)))
        throw new ArgumentNullException("source");

    return ConcatIterator<TSource>(source);
}

private static IEnumerable<TSource> ConcatIterator<TSource>(params IEnumerable<TSource>[] source)
{
    foreach (var enumerable in source)
    {
        foreach (var iteratorVariable in enumerable)
            yield return iteratorVariable;
    }
}

Playwriting

Answer 3

The Join method is like a SQL join, where the list are cross referenced based upon a condition, it isn't a string concatenation or Adding to a list. The Union method does do what you want, as does the Concat method, but both are LAZY evaluations, and have the requirement the parameters be non-null. They return either a ConcatIterator or a UnionIterator, and if called repeatedly this could cause problems. Eager evaluation results in different behavior, if that is what you want, then an extension method like the below could be used.

public static IEnumerable<T> myEagerConcat<T>(this IEnumerable<T> first,
                                                   IEnumerable<T> second)
{
    return (first ?? Enumerable.Empty<T>()).Concat(
           (second ?? Enumerable.Empty<T>())).ToList();
}

Lorna

Answer 4

Delayed invocation of the second and subsequent enumerables

I usually use Linq IEnumerable<T>.Concat() but today I needed to be 100% sure that the second enumeration was not enumerated until the first one has been processed until the end. (e.g. two db queries that I didn't want to run simultaneously). So the following function made the trick to delay the enumerations.

    IEnumerable<T> DelayedConcat<T>(params Func<IEnumerable<T>>[] enumerableList)
    {
        foreach(var enumerable in enumerableList)
        {
            foreach (var item in enumerable())
            {
                yield return item;
            }
        }
    }

Usage:

    return DelayedConcat(
                () => GetEnumerable1(),
                () => GetEnumerable2(),
 // and so on.. () => GetEnumerable3(),
                );

In this example GetEnumerable2 function invocation will be delayed until GetEnumerable1 has been enumerated till the end.

Ji

github.com/microsoft/referencesource/blob/master/System.Core/… Pretty much equal to the reference implementation – Tupungato

Incorrect @Tupungato . That one receives two enumerations, but mine receives two Func<IEnumerable<T>> and ensures you don't create the 2nd enumerable until the first one has been depleted. – Ji