Weird test coverage results for iterator block, why are these statements not executed?

Question

I'm using dotCover to analyze code coverage of my unit tests, and I'm getting some strange results... I have an iterator method for which the coverage is not complete, but the statements that are not covered are just the closing braces at the end of the method.

Here's the method I'm testing:

    public static IEnumerable<T> CommonPrefix<T>(
        this IEnumerable<T> source,
        IEnumerable<T> other,
        IEqualityComparer<T> comparer)
    {
        source.CheckArgumentNull("source");
        other.CheckArgumentNull("other");

        return source.CommonPrefixImpl(other, comparer);
    }

    private static IEnumerable<T> CommonPrefixImpl<T>(
        this IEnumerable<T> source,
        IEnumerable<T> other,
        IEqualityComparer<T> comparer)
    {
        comparer = comparer ?? EqualityComparer<T>.Default;

        using (IEnumerator<T> en1 = source.GetEnumerator(),
                              en2 = other.GetEnumerator())
        {
            while (en1.MoveNext() && en2.MoveNext())
            {
                if (comparer.Equals(en1.Current, en2.Current))
                    yield return en1.Current;
                else
                    yield break;
            }
        } // not covered
    } // not covered

The unit test:

    [Test]
    public void Test_CommonPrefix_SpecificComparer()
    {
        var first = new[] { "Foo", "Bar", "Baz", "Titi", "Tata", "Toto" };
        var second = new[] { "FOO", "bAR", "baz", "tata", "Toto" };

        var expected = new[] { "Foo", "Bar", "Baz" };
        var actual = first.CommonPrefix(second, StringComparer.CurrentCultureIgnoreCase);
        Assert.That(actual, Is.EquivalentTo(expected));
    }

And the coverage results:

I assume the closing brace of the using block is actually the calls to Dispose on the enumerators; but then, why is it not executed? I first suspected that NUnit wasn't disposing the enumerators, but I get the same result if I do a foreach on actual.

As for the second uncovered closing brace, I have no idea what it stands for... I guess it's related to how the compiler transforms the iterator block.

Can anyone shed some light on what these two "statements" are, and why they are not executed ?

---

EDIT: Peter raised a very good question: the results shown above were obtained when running the tests on a debug build. If I run the tests on a release build, coverage of the CommonPrefixImpl method is 100%, so it's probably related to compiler optimizations.

Scofield

Is this with a debug or release build? – Sawfish

@PeterRitchie, good thinking! I ran the tests on a debug build. I just tried again with a release build, and now the coverage is 100%. I'm still interested in an explanation, though... I don't know exactly what is different in a debug build. – Scofield

I provided an answer about how to instrument the code correctly so you don't get this problem, noting that binary instrumenters pretty much have a hard time doing this right, and provided a link to a tool that does the instrumentation correctly. A moderator deleted that answer. – Moor

@IraBaxter, yes, I see that; not sure why it was deleted. Anyway, thanks for your answer, but I'm not looking for another tool. The issue I described is not a major one (especially since it disappears with a release build), I was just curious to know why it happened. – Scofield

Answer 1

One of the problems with iterator methods is that the compiler generates a rather large and complex state machine to manage the deferred execution of the code within the iterator method. This usually generates a class or two. These classes are meant to deal with the general case and not your specific case, so there's likely at least a bit of code in there that is never used. You can look at what is generated by looking at your assembly with tools like ILSpy, JustDecompile or Reflector. It will show the classes in your assembly generated by the C# compiler (usually class names containing '<', etc.)

What the profiler knows about is how the PDB associates to your code and despite the possibility that all of the code you wrote possibly being executed, there's still a possibility that not all of the code generated by the compiler got executed. The profiler probably doesn't know this and simply says that a certain percentage (less than 100) of a particular iterator method got executed.

One of the things likely getting generated is exception handling code. Because the compiler doesn't know your code won't or possibly can't generate an exception it will still generate code to compensate for an exception--it needs to keep it's state from becoming corrupt. I bet if you included a way to throw an exception in various places in your iterator method based on some flag and ran the method twice (once without exceptions and once with exceptions in the same run) that the percentages would be different--likely higher because the generated exception handling code would then be exercised.

The fact that the end of the method "seems" to not be executed is likely because that code is part of a different method in the state machine that gets executed and the compiler never generates an association from that generated code to the code in your class.

UPDATE: to get a better understanding of what the compiler is doing and see an example of the type of code it generates see section 10.14 Iterators in the C# spec (http://www.microsoft.com/en-us/download/details.aspx?id=7029)

Sawfish

Thanks for the detailed answer! Do you have any idea why I get 100% coverage when I run the test on a release build? – Scofield

I can't be sure, but the compiler doesn't optimize as much in debug mode so it's a possibility that it optimizes away the code that it doesn't need. But, that's only one likely possibility. – Sawfish

Typically for code like this (i.e. empty constructors etc), you can add the System.Diagnostics.CodeAnalysis.ExcludeFromCodeCoverageAttribute. -- This auto generated code that's generated by the compiler, is there a way to force it to have this attribute so that it does not affect the code coverage results? – Argybargy

No, there is no way to force that attribute. dotCover should be smart enough to not profile compiler-generated code (there are generated code attributes on it though). Anything with '<' in the name is generated, easy to check. – Sawfish

Answer 2

In addition to your question and the in detail answer, I had the following behaviour.

    // less than 100% coverage
    public static IEnumerable<T> ForEachYieldDo<T>(this IEnumerable<T> source, Action<T> action)
    {
        foreach (var x in source)
        {
            action(x);
            yield return x;
        }
    }

    // 100% code coverage
    public static IEnumerable<T> ForEachSelectDo<T>(this IEnumerable<T> source, Action<T> action)
    {
        return source.Select(x =>
        {
            action(x);
            return x;
        });
    }

Both functions have the same behaviour. The action is only executed, if the item is processed. If the retrieval of the items is stopped, the action is not executed.

Cockswain

It turned out, that the first loop has 100% coverage when a test case gets ALL results for example using the .ToArray() method. – Cockswain