Expression tree - compile inner lambda in outer lambda - scoping resolution

Question

I am creating expression tree and there is a situation where I need to create one lambda in another lambda and store inner one in a class and add that class in expression tree. This is simple example of what I am trying to do (this code does not compile):

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Linq.Expressions;

namespace SimpleTest {
    public class LambdaWrapper {
        private Delegate compiledLambda;
        public LambdaWrapper(Delegate compiledLambda) {
            this.compiledLambda = compiledLambda;
        }
        public dynamic Execute() {
            return compiledLambda.DynamicInvoke();
        }
    }

    public class ForSO {

        public ParameterExpression Param;

        public LambdaExpression GetOuterLambda() {
            IList<Expression> lambdaBody = new List<Expression>();
            Param = Expression.Parameter(typeof(object), "Param");
            lambdaBody.Add(Expression.Assign(
                            Param, 
                            Expression.Constant("Value of 'param' valiable"))
                          );

            lambdaBody.Add(Expression.Call(
                            null, 
                            typeof(ForSO).GetMethod("Write"), 
                            Param)
                          );

            Delegate compiledInnerLambda = GetInnerLambda().Compile();
            LambdaWrapper wrapper = new LambdaWrapper(compiledInnerLambda);
            lambdaBody.Add(Expression.Constant(wrapper));
            //lambdaBody.Add(GetInnerLambda());
            return Expression.Lambda(
                        Expression.Block(
                                new ParameterExpression[] { Param }, 
                                lambdaBody));
        }

        public LambdaExpression GetInnerLambda() {
            return Expression.Lambda(
                    Expression.Block(
                        Expression.Call(null, 
                                typeof(ForSO).GetMethod("Write"), 
                                Expression.Constant("Inner lambda start")),
                        Expression.Call(null, 
                                typeof(ForSO).GetMethod("Write"), 
                                Param),
                        Expression.Call(null, 
                                typeof(ForSO).GetMethod("Write"), 
                                Expression.Constant("Inner lambda end"))
                    )
                );
        }

        public static void Write(object toWrite) {
            Console.WriteLine(toWrite);
        }

        public static void Main(string[] args) {
            ForSO so = new ForSO();
            LambdaWrapper wrapper = so.GetOuterLambda().Compile()
                                      .DynamicInvoke() as LambdaWrapper;
            wrapper.Execute();
            //(so.GetOuterLambda().Compile().DynamicInvoke() as Delegate).DynamicInvoke();
        }
    }
}

Problem is in GetInnerLambda().Compile() line in GetOuterLambda method. I am aware of one solution - it is in commented part of code. With that, everything works fine, but I need a wrapper as return value, not expression subtree (it might be ok to store inner lambda subtree in LambdaWrapper, and compile it later, but same problem occures).

Error I am getting is Unhandled Exception: System.InvalidOperationException: variable 'Param' of type 'System.Object' referenced from scope '', but it is not defined.

If I add Param to block variables in inner lambda, code compiles, but Param has not value assigned in outer lambda (and that makes sense).

How can this be solved?

Cimbri

Answer 1

With help from Balazs Tihanyi I found solution that works for me exactly as I need it. It is a bit more work because I had to create binders, but I my main project I already had them, so I created dummy binders for this example to work.

This is my final solution:

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Linq.Expressions;
using System.Reflection;
using System.Dynamic;


namespace SimpleTest {
    public class MyCreateBinder : CreateInstanceBinder {
        public MyCreateBinder(CallInfo info) : base(info) { }

        public override DynamicMetaObject FallbackCreateInstance(
                                        DynamicMetaObject target,
                                        DynamicMetaObject[] args,
                                        DynamicMetaObject errorSuggestion) {
            var param = args[0].Value;

            Type toCreate = target.Value as Type;
            var ctors = toCreate.GetConstructors()
                        .Where(c => c.GetParameters().Length == args.Length)
                        .ToArray();

            if (ctors.Length == 0)
                throw 
                    new Exception(
                      String.Format(
                      "Can not find constructor for '{0}' with {1} parameters",
                      toCreate, args.Length));
            ConstructorInfo ctorToUse = ctors[0];
            return new DynamicMetaObject(
                            Expression.New(
                                ctorToUse,
                                args.Select(a => a.Expression).ToList()),
                       BindingRestrictions.Empty);
        }
    }

    public class MySetMemberBinder : SetMemberBinder {

        public MySetMemberBinder(string name) : base(name, false) { }

        public override DynamicMetaObject FallbackSetMember(
                                DynamicMetaObject target,
                                DynamicMetaObject value,
                                DynamicMetaObject errorSuggestion) {

            throw new NotImplementedException();
        }
    }

    public class MyGetMemberBinder : GetMemberBinder {
        public MyGetMemberBinder(string name) : base(name, false) { }

        public override DynamicMetaObject FallbackGetMember(
                                        DynamicMetaObject target,
                                        DynamicMetaObject errorSuggestion) {
            throw new NotImplementedException();
        }
    }

    public class MyInvokeMemberBinder : InvokeMemberBinder {
        public MyInvokeMemberBinder(string name, CallInfo callInfo) 
            : base(name, false, callInfo) { }

        public override DynamicMetaObject FallbackInvokeMember(
                                    DynamicMetaObject target,
                                    DynamicMetaObject[] args,
                                    DynamicMetaObject errorSuggestion) {
            var a = this;
            throw new NotImplementedException();
        }

        public override DynamicMetaObject FallbackInvoke(
                                    DynamicMetaObject target,
                                    DynamicMetaObject[] args,
                                    DynamicMetaObject errorSuggestion) {
            throw new NotImplementedException();
        }
    }

    public class LambdaWrapper : IDynamicMetaObjectProvider {
        private Delegate compiledLambda;
        private LambdaExpression exp;

        public LambdaWrapper(LambdaExpression exp) {
            this.exp = exp;
            this.compiledLambda = exp.Compile();
        }
        public dynamic Execute(dynamic param) {
            return compiledLambda.DynamicInvoke(param);
        }

        public DynamicMetaObject GetMetaObject(Expression parameter) {
            return new MetaLambdaWrapper(parameter, this);
        }
    }

    public class MetaLambdaWrapper : DynamicMetaObject {
        public MetaLambdaWrapper(Expression parameter, object value) : 
            base(parameter, BindingRestrictions.Empty, value) { }

        public override DynamicMetaObject BindInvokeMember(
                                    InvokeMemberBinder binder,
                                    DynamicMetaObject[] args) {
            MethodInfo method = this.Value.GetType().GetMethod(binder.Name);
            return new DynamicMetaObject(
                        Expression.Call(
                            Expression.Constant(this.Value),
                                method,
                                    args.Select(a => a.Expression)),
                        BindingRestrictions.GetTypeRestriction(
                            this.Expression, 
                            typeof(LambdaWrapper)));
        }
    }


    public class ForSO {
        public ParameterExpression Param;
        public LambdaExpression GetOuterLambda() {
            Expression wrapper;
            IList<Expression> lambdaBody = new List<Expression>();
            Param = Expression.Parameter(typeof(object), "Param");
            lambdaBody.Add(Expression.Assign(
                            Param,
                            Expression.Constant("Value of 'param' variable"))
                          );
            lambdaBody.Add(Expression.Call(
                            null,
                            typeof(ForSO).GetMethod("Write"),
                            Param)
                          );

            wrapper = Expression.Dynamic(
                                new MyCreateBinder(new CallInfo(1)),
                                typeof(object),
                                Expression.Constant(typeof(LambdaWrapper)),
                                Expression.Quote(GetInnerLambda()));


            lambdaBody.Add(
                Expression.Dynamic(
                    new MyInvokeMemberBinder("Execute", new CallInfo(1)),
                    typeof(object),
                    wrapper,
                Expression.Constant("calling inner lambda from outer")));

            lambdaBody.Add(wrapper);

            return Expression.Lambda(
                        Expression.Block(
                                new ParameterExpression[] { Param },
                                lambdaBody));
        }

        public LambdaExpression GetInnerLambda() {
            ParameterExpression innerParam = Expression.Parameter(
                                                typeof(object), 
                                                "innerParam");
            return Expression.Lambda(
                    Expression.Block(
                        Expression.Call(null,
                                typeof(ForSO).GetMethod("Write"),
                                Expression.Constant("Inner lambda start")),
                        Expression.Call(null,
                                typeof(ForSO).GetMethod("Write"),
                                innerParam),
                        Expression.Call(null,
                                typeof(ForSO).GetMethod("Write"),
                                Param),
                        Expression.Call(null,
                                typeof(ForSO).GetMethod("Write"),
                                Expression.Constant("Inner lambda end"))
                    ),
                    innerParam
                );
        }

        public static void Write(object toWrite) {
            Console.WriteLine(toWrite);
        }

        public static void Main(string[] args) {
            Console.WriteLine("-----------------------------------");
            ForSO so = new ForSO();

            LambdaWrapper wrapper = (LambdaWrapper) so.GetOuterLambda()
                                                    .Compile()
                                                    .DynamicInvoke();
            Console.WriteLine("-----------------------------------");
            wrapper.Execute("Calling from main");
        }
    }

}

Cimbri

Answer 2

Well, since you can't use Param as a constant value in your inner lambda expression, I suggest you to add a lambda parameter to your expression:

public LambdaExpression GetInnerLambda()
{
    var param = Expression.Parameter(typeof(object));
    return Expression.Lambda(
        Expression.Block(
            Expression.Call(null,
                typeof(ForSO).GetMethod("Write"),
                Expression.Constant("Inner lambda start")),
            Expression.Call(null,
                typeof(ForSO).GetMethod("Write"),
                param),
            Expression.Call(null,
                typeof(ForSO).GetMethod("Write"),
                Expression.Constant("Inner lambda end"))
        ),
        param
    );
}

Then store the parameter's value in your LambdaWrapper class, and use it later as an argument in the DynamicInvoke call:

public class LambdaWrapper
{
    private object param;
    private Delegate compiledLambda;

    public LambdaWrapper(Delegate compiledLambda, object param)
    {
        this.compiledLambda = compiledLambda;
        this.param = param;
    }

    public dynamic Execute()
    {
        return compiledLambda.DynamicInvoke(param);
    }
}

That works, but the only issue is that it will call WriteLine on Param, which is a ParameterExpression object. To solve this, you have to create the wrapper class dynamically in your expression tree:

//lambdaBody.Add(Expression.Constant(wrapper));
lambdaBody.Add(Expression.New(
    typeof(LambdaWrapper).GetConstructor(new[] { typeof(Delegate), typeof(object) }),
    Expression.Constant(compiledInnerLambda),
    Param)
);

Then it will use the assigned value of Param. And since you don't use Param outside of GetOuterLambda, you can now use it as a local variable.

EDIT:

Here is my second attempt to solve this issue:

public LambdaExpression GetOuterLambda()
{
    ...
    //Delegate compiledInnerLambda = GetInnerLambda().Compile();
    //LambdaWrapper wrapper = new LambdaWrapper(compiledInnerLambda);

    lambdaBody.Add(Expression.New(
        typeof(LambdaWrapper).GetConstructor(new[] { typeof(Delegate) }),
        Expression.Call(
            Expression.Call(
                typeof(ForSO).GetMethod("GetInnerLambda", BindingFlags.Public | BindingFlags.Static),
                Param
            ),
            typeof(LambdaExpression).GetMethod("Compile", Type.EmptyTypes)
        )
    ));
    ...
}

public static LambdaExpression GetInnerLambda(object param)
{
    return Expression.Lambda(
        Expression.Block(
            Expression.Call(null,
                typeof(ForSO).GetMethod("Write"),
                Expression.Constant("Inner lambda start")),
            Expression.Call(null,
                typeof(ForSO).GetMethod("Write"),
                Expression.Constant(param)),
            Expression.Call(null,
                typeof(ForSO).GetMethod("Write"),
                Expression.Constant("Inner lambda end"))
        )
    );
}

This approach compiles this inner lambda when you run the outer delegate. By doing this, Param will be assigned before the inner lambda gets compiled.

Simplex

Thanks for the answer. What I don't like about this approach is that those lambdas are actual functions, and they can have their parameters (I didn't include that part in question because I don't have a problem with that), and Param is not only variable I need to access (there can be a lot of them), so I don't think that addig artificial parameters to resolve scoping is very elegant solution. – Cimbri

Updated answer might just work for me, but I'll have to check when I get back to my working computer. Thanks... – Cimbri

I checked if this works for me. Almost :). I took it one step further and created DynamicExpression for creating instance of LambdaWrapper. I had to create binders, so this soulution requires more work, but I already had them anyway in my main project. Thanks for the interest you showed in solving this issue :) – Cimbri