Can I generate an async method dynamically using System.Linq.Expressions?

Question

I know the compiler can't convert an async lambda expression to an expression tree, but is it possible to generate the expression tree manually ?

var expr = Expression.Lambda<Func<Task>>(
     // how do I use 'await' in the body here?
);
var func = expr.Compile();

I can't find any method related to async or await in the Expression class, but perhaps there's another way?

Hebraism

Answer 1

await involves significant compiler re-writing; the generated IL is quite dissimilar to the original C#, with variable hoisting (onto a class) and branching, tasks, continuations, etc. It certainly isn't something that can be represented in a simple lambda, although in recent versions of .NET with richer Expression support (Expression.Block etc), technically I suppose it is probably possible to mimic most of the things the compiler does for await - but you'd probably go quite loopy trying to do it by hand.

No, AFAIK, no facility to automate this translation exists in the Expression API, and frankly I wouldn't ever expect there to be.

The same probably could be say of ILGenerator; frankly, AFAIK the only "easy" way (and I use the word "easy" quite incorrectly) to use await in meta-programming would be to generate C# and run it through roslyn or CSharpCodeProvider.

Driblet

Thanks Marc. Unfortunately using Roslyn or CSharpCodeProvider isn't an option for me; I'm working on a Windows Store app, and the only codegen mechanism available on this platform (AFAIK) is Linq Expressions. I guess I will have to change my approach to avoid async code. – Hebraism

@ThomasLevesque or the other option, of course, is to embrace the instanity, perhaps writing an open source library that does all the Expression.Block work to make it work. I have a suspicion, however, that even this would not be enough, and that it is simply outside of what the Expression API is capable of representing. – Driblet

Well, maybe I'll give it a try someday if I'm bored ;) – Hebraism

I suspect it can be done, simply because the transforms applied to async/await are similar to those of custom iterators (yield), and the DLR, which is built atop System.Linq.Expressions is capable of generating that kind of code. Still, it'd be no small amount of work. The OP may be better off using Task.ContinueWith() in lieu of await. – Otolaryngology

There is alternative Expression tree implementations (FastExpressionCompiler), and there you may help to implement the support for async-await github.com/dadhi/FastExpressionCompiler/issues/47 – Hospodar

@MarcGravell Ping :) - I invite you to have a look at my answer. – Spoilsman

Answer 2

Late to the party, but I just published a lib which does accomplish this.

https://github.com/avonwyss/bsn.AsyncLambdaExpression

It leverages the fact that nested Lambdas in an Expression Tree can access the outer lambda variables (which are captured as closure, just like nested lambdas behave in C#). Thus the code creates a main lambda containing all the necessary variables and returning a Task, and a state machine in a nested lambda which is callable as continuation on awaiter.OnComplete(). Since the state, current awaiter etc. is stored in the closure, the state machine keeps its state even when the inner lambda stops execution.

The API to use it consists of two extension methods, one for awaiting and one for converting the Expression Tree with await to a lambda implementing the state machine and returning the Task.

Here's a code sample:

// Build a normal expression tree with "await" calls
var paraInput = Expression.Parameter(typeof(string), "input");
var exprTree = Expression.Lambda<Func<Task<string>, string>>(
        Expression.Block(
                Expression.Call(typeof(Task), nameof(Task.Delay), null, Expression.Constant(1000)).Await(false),
                paraInput.Await(false)),
        paraInput);

// Create compilable state machine async expression tree (result must be Task<?> or Task)
var asyncExprTree = exprTree.Async<Func<Task<string>, Task<string>>>();
var asyncCompiled = asyncExprTree.Compile();

// Invoke delegate as usual
var result = await asyncCompiled(Task.FromResult("test")).ConfigureAwait(false);

The DebugView of the original lambda Expression Tree is like this:

.Lambda #Lambda1<System.Func`2[System.Threading.Tasks.Task`1[System.String],System.String]>(System.Threading.Tasks.Task`1[System.String] $input)
{
    .Block() {
        .Call bsn.AsyncLambdaExpression.AsyncExpressionExtensions.AwaitVoid(.Call (.Call System.Threading.Tasks.Task.Delay(1000)).ConfigureAwait(False));
        .Call bsn.AsyncLambdaExpression.AsyncExpressionExtensions.AwaitResult(.Call $input.ConfigureAwait(False))
    }
}

The async conversion creates the following Expression Tree:

.Lambda #Lambda1<System.Func`2[System.Threading.Tasks.Task`1[System.String],System.Threading.Tasks.Task`1[System.String]]>(System.Threading.Tasks.Task`1[System.String] $input)
{
    .Block(
        System.Runtime.CompilerServices.ConfiguredTaskAwaitable+ConfiguredTaskAwaiter $awaiter,
        System.Runtime.CompilerServices.ConfiguredTaskAwaitable`1+ConfiguredTaskAwaiter[System.String] $awaiter,
        System.Int32 $state,
        System.Int32 $resumeState,
        System.Threading.Tasks.TaskCompletionSource`1[System.String] $taskCompletionSource,
        System.Action $continuation,
        System.Exception $exception) {
        $taskCompletionSource = .New System.Threading.Tasks.TaskCompletionSource`1[System.String](.Constant<System.Threading.Tasks.TaskCreationOptions>(RunContinuationsAsynchronously));
        $continuation = .Lambda #Lambda2<System.Action>;
        .Invoke $continuation();
        $taskCompletionSource.Task
    }
}

.Lambda #Lambda2<System.Action>() {
    .Try {
        .Loop  {
            .Switch ($state) {
            .Case (0):
                    .Block() {
                        $state = 1;
                        .If (
                            !($awaiter = .Call (.Call (.Call System.Threading.Tasks.Task.Delay(50)).ConfigureAwait(False)).GetAwaiter()).IsCompleted
                        ) {
                            .Block() {
                                .Call $awaiter.OnCompleted($continuation);
                                .Break :break { }
                            }
                        } .Else {
                            .Default(System.Void)
                        }
                    }
            .Case (1):
                    .Block() {
                        $state = 2;
                        .Call $awaiter.GetResult();
                        .If (
                            !($awaiter = .Call (.Call $input.ConfigureAwait(False)).GetAwaiter()).IsCompleted
                        ) {
                            .Block() {
                                .Call $awaiter.OnCompleted($continuation);
                                .Break :break { }
                            }
                        } .Else {
                            .Default(System.Void)
                        }
                    }
            .Case (2):
                    .Block(System.String $result:2) {
                        $result:2 = .Call $awaiter.GetResult();
                        .Call $taskCompletionSource.SetResult($result:2);
                        .Break :break { }
                    }
            .Default:
                    .Throw .New System.InvalidOperationException()
            }
        }
        .LabelTarget :break:
    } .Catch (System.Exception $ex) {
        .Call $taskCompletionSource.SetException($ex)
    }
}

This is a simple example; but the async conversion can not only handle await, but also the control flow expressions (loop, conditional, label/goto, try..catch..finally) in order to offer support for a wide variety of Expression Trees.

Spoilsman

Nice! I no longer need this, but I'll certainly look at it, just to see how you implemented it! – Hebraism