Wrap an IEnumerable and catch exceptions

Question

I've got a bunch of classes that can Process() objects, and return their own objects:

public override IEnumerable<T> Process(IEnumerable<T> incoming) { ... }

I want to write a processor class that can wrap one of these processors, and log any uncaught exceptions that the wrapped Process() method might throw. My first idea was something like this:

public override IEnumerable<T> Process(IEnumerable<T> incoming) {
    try {
        foreach (var x in this.processor.Process(incoming)) {
            yield return x;
        }
    } catch (Exception e) {
        WriteToLog(e);
        throw;
    }
}

but this doesn't work, due to CS1626: Cannot yield a value in the body of a try block with a catch clause.

So I want to write something that's conceptually equivalent but compiles. :-) I've got this:

public override IEnumerable<T> Process(IEnumerable<T> incoming) {
    IEnumerator<T> walker;
    try {
        walker = this.processor.Process(incoming).GetEnumerator();
    } catch (Exception e) {
        WriteToLog(e);
        throw;
    }

    while (true) {
        T value;
        try {
            if (!walker.MoveNext()) {
                break;
            }
            value = walker.Current;
        } catch (Exception e) {
            WriteToLog(e);
            throw;
        }
        yield return value;
    }
}

but that's more complex than I'd hoped, and I'm not entirely certain of either its correctness or that there isn't a much simpler way.

Am I on the right track here? Is there an easier way?

Kaela

What about implementing IEnumerable<T> directly and catching exceptions inside your MoveNext and Current? – Hackett

@Matt, I think that's the correct answer. You should have made it such. Nothing wrong with short and sweet. :) – Certain

Why not just remove the foreach loop and yield? – Psychopath

possible duplicate of LINQ query to perform a projection, skipping cases where the projection would cause an exception – Irradiant

The problem of the "equivalent" code is that the walker variable is not properly Disposeed. You should always Dispose IEnumerators since they may contain resources. – Ireland

Answer 1

A linq extension can be written to skip all elements that cause an exception and allow you to pass in an action to handle the exceptions that are raised.

public static IEnumerable<T> CatchExceptions<T> (this IEnumerable<T> src, Action<Exception> action = null) 
{
    using (var enumerator = src.GetEnumerator()) 
    {
        bool next = true;

        while (next) 
        {
            try 
            {
                next = enumerator.MoveNext();
            } 
            catch (Exception ex) 
            {
                if (action != null)
                    action(ex);
                continue;
            }

            if (next) 
                yield return enumerator.Current;
        }
    }
}

Example:

ienumerable.Select(e => e.something).CatchExceptions().ToArray()

ienumerable.Select(e => e.something).CatchExceptions((ex) => Logger.Log(ex, "something failed")).ToArray()

NOTE: This solution only catches exceptions raised when processing the originally yielded or projected value, so it is a great way to handle exceptions raised when you have chained IEnumerable expressions as OP has requested. If the exception is raised in the IEnumerable provider then there is a high chance the enumeration will be aborted, this method will handle the exception gracefully but there will be no way to resume the enumeration.

Exceptions raised in providers can occur commonly with file system or database readers as they are prone to timeouts and other runtime context issues. This fiddle shows how to create such a scenario: https://dotnetfiddle.net/a43Vtt You will have to handle those issues in the source directly.

Castleman

@ChrisSchaller, Enumerable.Range(-1,3).Select(v=>1/v).CatchExceptions().ToArray() will produce [-1,1]. – Landy

@Landy I thanks you good sir, that fiddle proves it, I will retract my previous comment, I must have had a different error when I tested, added proof here dotnetfiddle.net/BrWlRf – Shainashaine

I've edited this solution to include my notes, it is the best solution for OPs scenario and by editing I will upvote this as it is a superior answer to my own. It just can't be used to skip exceptions if they are raised by the enumeration source. – Shainashaine

@ChrisSchaller, sorry for the cold shower: the behavior on .Net Core 3 is not the same. See github.com/morelinq/MoreLINQ/issues/515#issuecomment-548330704 – Landy

Answer 2

If what you want to do is handle an exception during the processing of the result of an enumeration, then you try logic simply needs to go directly inside your for/while loop.

But your example reads as if you are trying to catch and skip over exceptions raised by the enumeration provider.

As far as I can ascertain, there is no way in C# to iterate over an enumerator and skip and exception that occurs within the enumerator itself. If the enumerator raises an exception, then all future calls to MoveNext() will result in false output.

The easiest way to explain why this happens is with this very simple enumerable:

IEnumerable<int> TestCases()
{
    yield return 1;
    yield return 2;
    throw new ApplicationException("fail eunmeration");
    yield return 3;
    yield return 4;
}

Understandably when we look at this example it is obvious that the thrown exception will cause this whole block to exit, and the 3rd and 4th yield statement will not ever be processed. In fact the get the usual 'Unreachable code detected' compiler warning on the 3rd yield statement.

So when the enumerable is a more complex, the same rules apply:

IEnumerable<int> TestCases2()
{
    foreach (var item in Enumerable.Range(0,10))
    {
        switch(item)
        {
            case 2:
            case 5:
                throw new ApplicationException("This bit failed");
            default:
                yield return item;
                break;
        }
    }
}

When the exception is raised, the processing of this block ceases and passes back up the call stack to the nearest exception handler.

ALL of the workable examples to get around this issue that I have found on SO do not proceed to the next item in the enumeration, they all break at the first exception.

Therefore to skip en exception in the enumeration you will need the provider to facilitate it. This is only possible really if your coded the provider, or you can contact the developer who did, the following is an over-simplified example of how you could achieve this:

IEnumerable<int> TestCases3(Action<int, Exception> exceptionHandler)
{
    foreach (var item in Enumerable.Range(0, 10))
    {
        int value = default(int);
        try
        {
            switch (item)
            {
                case 2:
                case 5:
                    throw new ApplicationException("This bit failed");
                default:
                    value = item;
                    break;
            }
        }
        catch(Exception e)
        {
            if (exceptionHandler != null)
            {
                exceptionHandler(item, e);
                continue;
            }
            else
                throw;
        }
        yield return value;
    }
}

...

foreach (var item in TestCases3(
    (int item, Exception ex) 
    => 
    Console.Out.WriteLine("Error on item: {0}, Exception: {1}", item, ex.Message)))
{
    Console.Out.WriteLine(item);
}

This will produce the following output:

0
1
Error on item: 2, Exception: This bit failed
3
4
Error on item: 5, Exception: This bit failed
6
7
8
9

I hope this clears up the issue for other developers in the future as it is a pretty common idea that we all get once we start getting deep into Linq and enumerations. Powerful stuff but there are some logical limitations.

Shainashaine

Answer 3

It might be nicer if the process object was something that processed just one item in the list at a time (if that's even possible), that way you could collect each individual exception and return an AggregateException just like the Task Parallel library does.

[If process operates on the list as a whole, or if a single exception needs to abort the whole process obviously this suggestion isn't appropriate.]

Incorporeal

It might be nicer, or it might not, but that's a (semi-external) interface I can't change, at least not on a reasonable timescale. :-) – Kaela

Answer 4

Instead of solving your specific problem only, you can have these helper functions in your toolkit:

    static IEnumerable<T> RunEnumerator<T>(Func<IEnumerator<T>> generator, 
        Action<Exception> onException)
    {
        using (var enumerator = generator())
        {
            if (enumerator == null) 
                yield break;
            for (; ; )
            {
                //Avoid creating a default value with
                //unknown type T, as we don't know is it possible to do so
                T[] value = null;
                try
                {
                    if (enumerator.MoveNext())
                        value = new T[] { enumerator.Current };
                }
                catch (Exception e)
                {
                    onException(e);
                }
                if (value != null)
                    yield return value[0];
                else
                    yield break;
            }
        }
    }

    public static IEnumerable<T> WithExceptionHandler<T>(this IEnumerable<T> orig, 
        Action<Exception> onException)
    {
        return RunEnumerator(() =>
        {
            try
            {
                return orig.GetEnumerator();
            }
            catch (Exception e)
            {
                onException(e);
                return null;
            }
        }, onException);
    }
}

WithExceptionHandler<T> converts an IEnumerable<T> into another, and when exception occurs, the onException callback will be called. In this way you can have your process function implemented like this:

public override IEnumerable<T> Process(IEnumerable<T> incoming) {
    return incoming.WithExceptionHandler(e => {
        WriteToLog(e);
        throw;
    }
}

In this way, you can also do something else like ignore the exception completely and let the iteration simply stops. You may also adjust it a little bit by used a Func<Exception, bool> instead of Action<Exception> and allows the exception callback to decide shall the iteration continue or not.

Ireland

This won't work as expected, if enumerator.MoveNext() throws an exception, the enumerator is effectively dead, you will never get to the next item from the enumerator in this way. Your iteration will always stop at the first exception, so your wrapper hasn't achieved anything more than if you had wrapped the original processing loop in a try catch block. – Shainashaine

Well, this depends on how your enumerator being implemented. You can easily write your own IEnumerator that bypassing this limit. Although you cannot rely on any other system ones, at least you can use for your own code. – Ireland

Don't get me wrong, I am truly interested in a solution like this. I'd love to see an example that works :) But if MoveNext() throws an exception it is toast, if MoveNext handles the exception, then MoveNext is going to return true, so all this wrappering is totally unnecessary. That's what I'm getting at, this wrapper will never 'skip' the exception, it will handle the first one and exit gracefully, but it will never get to the next item after the first exception. – Shainashaine

Answer 5

The loop and yield are not needed (at least in your example). Simple remove them and there is no longer a restriction on catch blocks.

public override IEnumerable<T> Process(IEnumerable<T> incoming) {
    try {
        return this.processor.Process(incoming);
    } catch (Exception e) {
        WriteToLog(e);
        throw;
    }
}

I am assuming this.processor.Process(incoming) returns a collection implementing IEnumerable<T>, so therefore you do not need to create a new iterator. If this.processor.Process(incoming) is lazy evaluating then

public override IEnumerable<T> Process(IEnumerable<T> incoming) {
    try {
        return this.processor.Process(incoming).ToList();
    } catch (Exception e) {
        WriteToLog(e);
        throw;
    }
}

Psychopath

where does it say this.processor.Process(incoming) is lazy? – Psychopath

@Bear Monkey, I will rephrase. If the exception happens upon invocation of GetEnumerator() then your code will not catch it because it never invokes GetEnumerator(). – Certain

@Bear Monkey, what doesn't make sense? – Certain

@Kirk i know what your talking about but your assuming that this.processor.Process(incoming) is lazy just because its caller is returning IEnumerable<T>. Just because its returning IEnumerable<T> doesnt mean it has to be lazy at all. – Psychopath

@Bear Monkey, you misunderstand. The entire contract of IEnumerable is lazy -- that's what it means for it to have a GetEnumerator() method that returns an IEnumerator. Therefore all usages of IEnumerable are automatically lazy. Your update to your answer will technically work, but it somewhat ruins the pull-as-you-go contract of an IEnumerable. Since Matt's suggestion would not exhibit that limitation, I prefer his answer. – Certain

@Kirk No that is not true. By your definition a List<T> is always lazy evaluated because it implements IEnumerable<T> which it clearly is not. He wanted an easier solution so i gave him one. – Psychopath

@Bear Monkey, even a List<T> is technically evaluated lazily. Not until you invoke GetEnumerator() has anything happened (such as instantiating the enumerator) -- before then you are simply passing around references. – Certain

@Kirk err no... List<T> is not evaluated lazily. List<T> evaluation is not performed by its GetEnumerator() all that does is return an IEnumerator<T> to iterate over the collection. – Psychopath

@kirk The IEnumerator<T> returned by List<T>.GetEnumerator() is not lazy evaluating the list. All its does is to iterate over the collection. Other collections may lazy evaluate when you iterate over them but List<T> isn't one of them. – Psychopath

-1 This really wont work, see https://mcmap.net/q/587113/-linq-query-to-perform-a-projection-skipping-or-wrapping-exceptions-where-source-throws-on-ienumerable-getnext for one that does – Irradiant

Creating an IEnumerable usually doesn't throw, so you won't catch any exception and return (before the first item is yielded). The consumer will be happy, starts iterating and hit the exception - that's why this does not work. – Hoon