Why does the async keyword generate an enumerator & additional struct when compiled?

Question

If I create a simple class like the following:

public class TestClass
{
    public Task TestMethod(int someParameter)
    {
        return Task.FromResult(someParameter);
    }

    public async Task TestMethod(bool someParameter)
    {
        await Task.FromResult(someParameter);
    }
}

and examine it within NDepend, it shows that the TestMethod taking a bool and being async Task has a struct generated for it with an enumerator, the enumerator state machine and some additional stuff.

Why does the compiler generate a struct called TestClass+<TestMethod>d__0 with an enumerator for the async method?

It seems to generate more IL than what the actual method produces. In this example, the compiler generates 35 lines of IL for my class, while it generates 81 lines of IL for the struct. It's also increasing the complexity of the compiled code and causing NDepend to flag it for several rule violations.

Broida

The code in async/await is based on Jeff Richter's Async Enumerator pattern. There is already a voice suggesting a fix for this ndepend.uservoice.com/forums/226344-ndepend-user-voice/…. – Unwearied

Because that's how async/await works. What you see is the implementation detail of the feature. – Predestination

This really underlines the importance of making sure you're not blindly using async all over the place when it's not needed. That's more overhead than what I guess I was expecting. – Broida

@JohnathonSullinger Use async only when you need it. – Predestination

@Aron: I could be wrong, but I don't think async is based on Async Enumerator. While they appear similar at first glance, async is capable of much more complex transformations. – Argufy

@StephenCleary much much much more complex yes. But creating an enumerator to store state of a function between yielding control back to the caller. – Unwearied

Answer 1

This is because the async and await keywords are just syntactical sugar for something called coroutines.

There are no special IL instructions to support the creation of asynchronous methods. Instead, an async method can be seen as kind of a state machine somehow.

I will try to make this example as short as possible:

[TestClass]
public class AsyncTest
{
    [TestMethod]
    public async Task RunTest_1()
    {
        var result = await GetStringAsync();
        Console.WriteLine(result);
    }

    private async Task AppendLineAsync(StringBuilder builder, string text)
    {
        await Task.Delay(1000);
        builder.AppendLine(text);
    }

    public async Task<string> GetStringAsync()
    {
        // Code before first await
        var builder = new StringBuilder();
        var secondLine = "Second Line";

        // First await
        await AppendLineAsync(builder, "First Line");

        // Inner synchronous code
        builder.AppendLine(secondLine);

        // Second await
        await AppendLineAsync(builder, "Third Line");

        // Return
        return builder.ToString();
    }
}

This is some async code as you've probably become used to: Our GetStringAsync method at first creates a StringBuilder synchronously, then it awaits some asynchronous methods and finally it returns the result. How would this be implemented if there was no await keyword?

Add the following code to the AsyncTest class:

[TestMethod]
public async Task RunTest_2()
{
    var result = await GetStringAsyncWithoutAwait();
    Console.WriteLine(result);
}

public Task<string> GetStringAsyncWithoutAwait()
{
    // Code before first await
    var builder = new StringBuilder();
    var secondLine = "Second Line";

    return new StateMachine(this, builder, secondLine).CreateTask();
}

private class StateMachine
{
    private readonly AsyncTest instance;
    private readonly StringBuilder builder;
    private readonly string secondLine;
    private readonly TaskCompletionSource<string> completionSource;

    private int state = 0;

    public StateMachine(AsyncTest instance, StringBuilder builder, string secondLine)
    {
        this.instance = instance;
        this.builder = builder;
        this.secondLine = secondLine;
        this.completionSource = new TaskCompletionSource<string>();
    }

    public Task<string> CreateTask()
    {
        DoWork();
        return this.completionSource.Task;
    }

    private void DoWork()
    {
        switch (this.state)
        {
            case 0:
                goto state_0;
            case 1:
                goto state_1;
            case 2:
                goto state_2;
        }

        state_0:
            this.state = 1;

            // First await
            var firstAwaiter = this.instance.AppendLineAsync(builder, "First Line")
                                        .GetAwaiter();
            firstAwaiter.OnCompleted(DoWork);
            return;

        state_1:
            this.state = 2;

            // Inner synchronous code
            this.builder.AppendLine(this.secondLine);

            // Second await
            var secondAwaiter = this.instance.AppendLineAsync(builder, "Third Line")
                                            .GetAwaiter();
            secondAwaiter.OnCompleted(DoWork);
            return;

        state_2:
            // Return
            var result = this.builder.ToString();
            this.completionSource.SetResult(result);
    }
}

So obviously the code before the first await keyword just stays the same. Everything else is converted to a state machine which uses goto statements to execute your previous code piecewise. Every time one of the awaited tasks is completed, the state machine advances to the next step.

This example is oversimplified to clarify what happens behind the scenes. Add error handling and some foreach-Loops in your async method, and the state machine gets much more complex.

By the way, there is another construct in C# that does such a thing: the yield keyword. This also generates a state machine and the code looks quite similar to what await produces.

For further reading, look into this CodeProject which takes a deeper look into the generated state machine.

Pudding

Thanks for the details answer, this really helped make it clear. I will be sure to check out the link as well – Broida

In this example, your DoWork() method is a simplified variation of .MoveNext() in the async state machine right? – Broida

That's right. Minus optimization, cancellation, exception handling, internal framework methods. – Pudding

Answer 2

The original code generation for async was closely related to that of enumerator blocks, so they started off using the same code in the compiler for those two code transformations. It has changed quite a bit since then, but it still has some holdovers from the original design (such as the name MoveNext).

For more about the compiler-generated parts, Jon Skeet's blog series is the best source.

Argufy

Alternate (newer) link, in case the old one breaks at some point: codeblog.jonskeet.uk/2011/05/08/eduasync-part-1-introduction – Nsf