LINQ query to perform a projection, skipping or wrapping exceptions where source throws on IEnumerable.GetNext()

Question

I'd like a general solution but as an example, assume i have an IEnumerable<string>, where some can be parsed as integers, and some cannot.

var strings = new string[] { "1", "2", "notint", "3" };

Obviously if i did Select(s => int.Parse(s, temp)) it'd throw an exception when enumerated.

In this case i could do .All(s => int.TryParse(s, out temp)) first, however i want a general solution where i don't have to enumerate the IEnumerable twice.

Ideally i'd like to be able to do the following, which calls my magic exception skipping method:

// e.g. parsing strings
var strings = new string[] { "1", "2", "notint", "3" };
var numbers = strings.Select(s => int.Parse(s)).SkipExceptions();
// e.g. encountering null object
var objects = new object[] { new object(), new object(), null, new object() }
var objecttostrings = objects.Select(o => o.ToString()).SkipExceptions();
// e.g. calling a method that could throw
var myClassInstances = new MyClass[] { new MyClass(), new MyClass(CauseMethodToThrow:true) };
var myClassResultOfMethod = myClassInstances.Select(mci => mci.MethodThatCouldThrow()).SkipExceptions();

How can i write the SkipExceptions() extension method?

---

Some great answers for a SelectSkipExceptions() method, however i wonder if a SkipExceptions() method could be created, along the same lines as AsParallel().

Tolbutamide

Inferior cousin question I'm voting to close: #3836133 – Sworn

Answer 1

How about this (you might want to give this special Select Extension a better name)

public static IEnumerable<TOutput> SelectIgnoringExceptions<TInput, TOutput>(
    this IEnumerable<TInput> values, Func<TInput, TOutput> selector)
   {
        foreach (var item in values)
        {
            TOutput output = default(TOutput);

            try
            {
                output = selector(item);
            }
            catch 
            {
                continue;
            }

            yield return output;
        }
    }

Edit5 Added a using statement, thanks for the suggestion in comments

    public static IEnumerable<T> SkipExceptions<T>(
        this IEnumerable<T> values)
    {
        using(var enumerator = values.GetEnumerator())
        {
           bool next = true;
           while (next)
           {
               try
               {
                   next = enumerator.MoveNext();
               }
               catch
               {
                   continue;
               }

               if(next) yield return enumerator.Current;
           } 
        }
    }

However this relies on the incoming IEnumerable not already being created (and therefore already having thrown Exceptions) as a list by the preceding Function. E.g. this would probably not work if you call it like this: Select(..).ToList().SkipExceptions()

Balmy

I thought of the same approach for SkipExceptions, but I didn't think it could actually work... well done! – Dishpan

Yeah, its always surprising to re-learn that Linq actually starts with the "outermost" function first on IEnumerables, so you can get stuff like that actually work. – Balmy

BTW, this will only work with streaming operators (like Where, Select, etc), not with buffering operators (OrderBy, GroupBy, etc). For instance, if you insert an OrderBy between Select and SkipExceptions, it will return an empty sequence. – Dishpan

Yes, one should probably add buffering operators beforehand (so to speak) of SkipExceptions, like so: Select(...).SkipExceptions(...).GroupBy(...) - that is basically what i meant in my confusing worded last sentence in the original post. There is no way of finding out what has already been done with the IEnumerable when it gets passed into SkipExceptions, so there is no way around that limitation – Balmy

you could add an optional parameter to log exceptions if desired. See my answer to a similar question here – Twinkling

One word from practise: skipping exceptions when enumerating is nice but there is a risk of endless loops: At work we applied this on some enumerator that raised an error in MoveNext before its internal pointer moved on to the next item. This resulted in some endless loop... Good MoveNext implementations should never raise exceptions before advancing their internal pointer to next element or to "end of list / null". But there are bad implementations out there. – Five

Answer 2

Create a TryParseInt method that returns a Nullable<int>:

int? TryParseInt(string s)
{
    int i;
    if (int.TryParse(s, out i))
        return i;
    return null;
}

And use it in your query like that:

var numbers = strings.Select(s => TryParseInt(s))
                     .Where(i => i.HasValue)
                     .Select(i => i.Value);

See also this article by Bill Wagner, which presents a very similar case.

---

Now, i don't think you can write something like a generic SkipExceptions method, because you would catch the exception too late, and it would end the Select loop... But you could probably write a SelectSkipException method:

public static IEnumerable<TResult> SelectSkipExceptions<TSource, TResult>(
    this IEnumerable<TSource> source,
    Func<TSource, TResult> selector)
{
    if (source == null)
        throw new ArgumentNullException("source");
    if (selector == null)
        throw new ArgumentNullException("selector");
    return source.SelectSkipExceptionsIterator(selector);
}

private static IEnumerable<TResult> SelectSkipExceptionsIterator<TSource, TResult>(
    this IEnumerable<TSource> source,
    Func<TSource, TResult> selector)
{
    foreach(var item in source)
    {
        TResult value = default(TResult);
        try
        {
            value = selector(item);
        }
        catch
        {
            continue;
        }
        yield return value;
    }
}

Dishpan

I'd like a general solution, i'd like to avoid different methods for different ways of generating an exception. The strings to ints was just an example. I'd guess something could be created that was like a select, except that it wrapped the lambda in a try...catch (saving the result) and only did yield if no exception. – Tolbutamide

@Thomas, @Urban, I think it might be possible to write a SkipExceptions() method, i think this because of AsParallel()'s existence (although i don't know about how it works it does seem to alter the way the methods/query 'before' it runs). – Tolbutamide

@George I added my attempt of a SkipExceptions method. Have a look on it if you like. – Balmy

Answer 3

Even the accepted answer may not be "general" enough. What if some day you find that you need to know what exceptions occurred?

The following extension

static class EnumeratorHelper {

    //Don't forget that GetEnumerator() call can throw exceptions as well.
    //Since it is not easy to wrap this within a using + try catch block with yield,
    //I have to create a helper function for the using block.
    private static IEnumerable<T> RunEnumerator<T>(Func<IEnumerator<T>> generator, 
        Func<Exception, bool> onException)
    {
        using (var enumerator = generator())
        {
            if (enumerator == null) 
                yield break;
            for (; ; )
            {
                //You don't know how to create a value of T,
                //and you don't know weather it can be null,
                //but you can always have a T[] with null value.
                T[] value = null;
                try
                {
                    if (enumerator.MoveNext())
                        value = new T[] { enumerator.Current };
                }
                catch (Exception e)
                {
                    if (onException(e))
                        continue;
                }
                if (value != null)
                    yield return value[0];
                else
                    yield break;
            }
        }
    }

    public static IEnumerable<T> WithExceptionHandler<T>(this IEnumerable<T> orig, 
        Func<Exception, bool> onException)
    {
        return RunEnumerator(() =>
        {
            try
            {
                return orig.GetEnumerator();
            }
            catch (Exception e)
            {
                onException(e);
                return null;
            }
        }, onException);
    }

}

will help. Now you can add SkipExceptions:

 public static IEnumerable<T> SkipExceptions<T>(this IEnumerable<T> orig){
     return orig.WithExceptionHandler(orig, e => true);
 }

By using different onException callback, you can do different things

• Break the iteration but ignore the exception: e => false
• Try to continue iteration: e => true
• Log the exception, etc

Noles

Answer 4

Here's a small complete program to demonstrate an answer inspired by the maybe monad. You might want to change the name of the 'Maybe' class, as it is inspired by rather than actually being a 'Maybe' as defined in other languages.

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;

namespace TestMaybe
{
    class Program
    {
        static void Main(string[] args)
        {
            var strings = new string[] { "1", "2", "notint", "3" };
            var ints = strings.Select(s => new Maybe<string, int>(s, str => int.Parse(str))).Where(m => !m.nothing).Select(m => m.value);
            foreach (var i in ints)
            {
                Console.WriteLine(i);
            }
            Console.ReadLine();

        }
    }

    public class Maybe<T1, T2>
    {
        public readonly bool nothing;
        public readonly T2 value;

        public Maybe(T1 input, Func<T1, T2> map)
        {
            try
            {
                value = map(input);
            }
            catch (Exception)
            {
                nothing = true;
            }            
        }
    }
}

Edit: depending on the needs of your code, you might also want nothing set to true if the result of map(input) is null.

Loraine

Answer 5

This is the same answer as Thomas's, but with a lambda & LINQ expression. +1 for Thomas.

Func<string, int?> tryParse = s =>
{
    int? r = null;
    int i;
    if (int.TryParse(s, out i))
    {
        r = i;
    }
    return r;
};

var ints =
    from s in strings
    let i = tryParse(s)
    where i != null
    select i.Value;

Protocol

Answer 6

You could just chain the Where and Select method together.

var numbers = strings.Where(s =>
                      {
                          int i;
                          return int.TryParse(s, out i);
                      }).Select(int.Parse);

The use of the Where method effectively removes the need for you to write your own SkipExceptions method, because this is basically what you are doing.

Notepaper

That does solve issue, however to create a general solution you'd have to wrap the select lambda in a try...catch in the where method, and write it again in the select method. It would still evaluate the lambda twice for each element. – Tolbutamide

The question now (even if it didnt originally), asks for a general solution. I agree with the -1 and think most of the other 0-voted ones belong in that camp. But I'm feeling nice so I'll ask you to delete the answer rather than sending you on the way to a Peer Pressure badge (and same for the others). – Sworn

"your feeling nice" thank's for your benediction oh-holy-one-of-14k-points, I will sleep tonight – Notepaper

+1 I would say this is a perfectly valid solution for anybody else stumbling on this question. Its clearer what is going on than a generic "swallow exceptions" extension method. It's also less prone to debug issues arising from exceptions that you didn't anticipate. In the real world I would much rather see this in somebody's code then what is written in the accepted answer. – Guglielma