How to create LINQ Expression Tree to select an anonymous type

Question

I would like to generate the following select statement dynamically using expression trees:

var v = from c in Countries
        where c.City == "London"
        select new {c.Name, c.Population};

I have worked out how to generate

var v = from c in Countries
        where c.City == "London"
        select new {c.Name};

but I cannot seem to find a constructor/overload that will let me specify multiple properties in my select lambda.

Trieste

Answer 1

This can be done, as mentioned, with the help of Reflection Emit and a helper class I've included below. The code below is a work in progress, so take it for what it's worth... 'it works on my box'. The SelectDynamic method class should be tossed in a static extension method class.

As expected, you won't get any Intellisense since the type isn't created until runtime. Works good on late-bound data controls.

public static IQueryable SelectDynamic(this IQueryable source, IEnumerable<string> fieldNames)
{
    Dictionary<string, PropertyInfo> sourceProperties = fieldNames.ToDictionary(name => name, name => source.ElementType.GetProperty(name));
    Type dynamicType = LinqRuntimeTypeBuilder.GetDynamicType(sourceProperties.Values);

    ParameterExpression sourceItem = Expression.Parameter(source.ElementType, "t");
    IEnumerable<MemberBinding> bindings = dynamicType.GetFields().Select(p => Expression.Bind(p, Expression.Property(sourceItem, sourceProperties[p.Name]))).OfType<MemberBinding>();

    Expression selector = Expression.Lambda(Expression.MemberInit(
        Expression.New(dynamicType.GetConstructor(Type.EmptyTypes)), bindings), sourceItem);

    return source.Provider.CreateQuery(Expression.Call(typeof(Queryable), "Select", new Type[] { source.ElementType, dynamicType },
                 Expression.Constant(source), selector));
}



public static class LinqRuntimeTypeBuilder
{
    private static readonly ILog log = LogManager.GetLogger(System.Reflection.MethodBase.GetCurrentMethod().DeclaringType);
    private static AssemblyName assemblyName = new AssemblyName() { Name = "DynamicLinqTypes" };
    private static ModuleBuilder moduleBuilder = null;
    private static Dictionary<string, Type> builtTypes = new Dictionary<string, Type>();

    static LinqRuntimeTypeBuilder()
    {
        moduleBuilder = Thread.GetDomain().DefineDynamicAssembly(assemblyName, AssemblyBuilderAccess.Run).DefineDynamicModule(assemblyName.Name);
    }

    private static string GetTypeKey(Dictionary<string, Type> fields)
    {
        //TODO: optimize the type caching -- if fields are simply reordered, that doesn't mean that they're actually different types, so this needs to be smarter
        string key = string.Empty;
        foreach (var field in fields)
            key += field.Key + ";" + field.Value.Name + ";";

        return key;
    }

    public static Type GetDynamicType(Dictionary<string, Type> fields)
    {
        if (null == fields)
            throw new ArgumentNullException("fields");
        if (0 == fields.Count)
            throw new ArgumentOutOfRangeException("fields", "fields must have at least 1 field definition");

        try
        {
            Monitor.Enter(builtTypes);
            string className = GetTypeKey(fields);

            if (builtTypes.ContainsKey(className))
                return builtTypes[className];

            TypeBuilder typeBuilder = moduleBuilder.DefineType(className, TypeAttributes.Public | TypeAttributes.Class | TypeAttributes.Serializable);

            foreach (var field in fields)                    
                typeBuilder.DefineField(field.Key, field.Value, FieldAttributes.Public);

            builtTypes[className] = typeBuilder.CreateType();

            return builtTypes[className];
        }
        catch (Exception ex)
        {
            log.Error(ex);
        }
        finally
        {
            Monitor.Exit(builtTypes);
        }

        return null;
    }


    private static string GetTypeKey(IEnumerable<PropertyInfo> fields)
    {
        return GetTypeKey(fields.ToDictionary(f => f.Name, f => f.PropertyType));
    }

    public static Type GetDynamicType(IEnumerable<PropertyInfo> fields)
    {
        return GetDynamicType(fields.ToDictionary(f => f.Name, f => f.PropertyType));
    }
}

Culverin

wonderful, didn't know creating a type at runtime was that easy!thanks! – Urata

Good but "Cannot serialize interface System.Linq.IQueryable" – Viridescent

You can put OrderBy in your //TODO for optimization and its done. – Aboulia

@Ethan J. Brown, could you tell me how to modify your code if the source is IEnumerable instead of IQueryable? Thanks! – Belfort

I've been using this (well, similar) and have been getting an Unable to create a constant value of type error. I've fixed this by replacing Expression.Constant(source) with source.Expression on the last line. Hope this helps someone :) – Andry

@Ethan J. Brown, Thanks for your great answer, How can I use this code with nested types, for example x=> new { a=1, b = new { ba=1, bb=2} }; – Affricate

I like this type builder, but calling .GetProperties() on the new type returns an empty array. To get those, is one required to go with the more complex solution at this question? https://mcmap.net/q/108799/-how-to-dynamically-create-a-class – Ouachita

Answer 2

The accepted answer is very useful, but I needed something a little closer to a real anonymous type.

A real anonymous type has read-only properties, a constructor for filling in all of the values, an implementation of Equals/GetHashCode for comparing the values of each property, and an implementation ToString that includes the name/value of each property. (See https://msdn.microsoft.com/en-us/library/bb397696.aspx for a full description of anonymous types.)

Based on that definition of anonymous classes, I put a class that generates dynamic anonymous types on github at https://github.com/dotlattice/LatticeUtils/blob/master/LatticeUtils/AnonymousTypeUtils.cs. The project also contains some unit tests to make sure the fake anonymous types behave like real ones.

Here's a very basic example of how to use it:

AnonymousTypeUtils.CreateObject(new Dictionary<string, object>
{
    { "a", 1 },
    { "b", 2 }
});

Also, another note: I found that when using a dynamic anonymous type with Entity Framework, the constructor must be called with the "members" parameter set. For example:

Expression.New(
    constructor: anonymousType.GetConstructors().Single(), 
    arguments: propertyExpressions,
    members: anonymousType.GetProperties().Cast<MemberInfo>().ToArray()
); 

If you used one of the versions of Expression.New that does not include the "members" parameter, Entity Framework would not recognize it as the constructor of an anonymous type. So I assume that means a real anonymous type's constructor expression would include that "members" information.

Extragalactic

Answer 3

Maybe a bit late but may help to someone.

You Can generate dynamic select by call DynamicSelectGenerator in select from an entity.

public static Func<T, T> DynamicSelectGenerator<T>()
            {
                // get Properties of the T
                var fields = typeof(T).GetProperties().Select(propertyInfo => propertyInfo.Name).ToArray();

            // input parameter "o"
            var xParameter = Expression.Parameter(typeof(T), "o");

            // new statement "new Data()"
            var xNew = Expression.New(typeof(T));

            // create initializers
            var bindings = fields.Select(o => o.Trim())
                .Select(o =>
                {

                    // property "Field1"
                    var mi = typeof(T).GetProperty(o);

                    // original value "o.Field1"
                    var xOriginal = Expression.Property(xParameter, mi);

                    // set value "Field1 = o.Field1"
                    return Expression.Bind(mi, xOriginal);
                }
            );

            // initialization "new Data { Field1 = o.Field1, Field2 = o.Field2 }"
            var xInit = Expression.MemberInit(xNew, bindings);

            // expression "o => new Data { Field1 = o.Field1, Field2 = o.Field2 }"
            var lambda = Expression.Lambda<Func<T, T>>(xInit, xParameter);

            // compile to Func<Data, Data>
            return lambda.Compile();
        }

And use by this code:

var result = dbContextInstancs.EntityClass.Select(DynamicSelectGenerator<EntityClass>());

Linville

Expression.New(typeof(T)) It will not work if T is one of entity mapped types. – Koodoo

Answer 4

I don't believe that you will be able to achieve this. Although when you do select new { c.Name, c.Population } it seems like you're not creating a class you actually are. If you have a look at the compiled output in Reflector or the raw IL you will be able to see this.

You'll have a class which would look something like this:

[CompilerGenerated]
private class <>c__Class {
  public string Name { get; set; }
  public int Population { get; set; }
}

(Ok, I cleaned it up a touch, since a property is really just a get_Name() and set_Name(name) method set anyway)

What you're trying to do is proper dynamic class creation, something which wont be available until .NET 4.0 comes out (and even then I'm not really sure if it'll be able to achieve what you want).

You're best solution would be to define the different anonymous classes and then have some kind of logical check to determine which one to create, and to create it you can use the object System.Linq.Expressions.NewExpression.

But, it may be (in theory at least) possible to do it, if you're getting really hard-core about the underlying LINQ provider. If you are writing your own LINQ provider you can detect if the currently-parsed expression is a Select, then you determine the CompilerGenerated class, reflect for its constructor and create.

Defiantly not a simple task, but it would be how LINQ to SQL, LINQ to XML, etc all do it.

Sextillion

Good summary. Pity there's no way to generate an expression to generate a new type. Although, I can imagine that opens up a big can of worms. :) – Shielashield

I will check how the extensions in System.Linq.Dynamic work, I am guessing there must be a way to do this if this class can do it. – Trieste

Answer 5

You could use the IQueryable-Extensions here, which is an implemantation of the solution described by "Ethan J. Brown":

https://github.com/thiscode/DynamicSelectExtensions

The Extension builds dynamically an anonymous type.

Then you can do this:

var YourDynamicListOfFields = new List<string>(

    "field1",
    "field2",
    [...]

)
var query = query.SelectPartially(YourDynamicListOfFields);

Bridgetbridgetown

Answer 6

You could use a parameter class instead of working with an anonymous type. In your example you can create a parameter class like this:

public struct ParamClass {
    public string Name { get; set; };
    public int Population { get; set; };
}

…and put it into your select like this:

var v = from c in Countries
        where c.City == "London"
        select new ParamClass {c.Name, c.Population};

What you get out is something of the type IQueryable<ParamClass>.

Bewilderment

Answer 7

This compiles, I dunno if it works however...

myEnumerable.Select((p) => { return new { Name = p.Name, Description = p.Description }; });

Assuming p is what your transforming, and the select statement is returning an anon type, using the function declaration of lambda's.

Edit: I also don't know how you would generate this dynamically. But at least it shows you how to use the select lambda to return an anon type with multiple values

Edit2:

You would also have to bare in mind, that the c# compiler actually generates static classes of the anon type. So the anon type does actually have a type after compile time. So if your generating these queries at run time (which I assume you are) you may have to construct a type using the various reflection methods (I believe you can use them to make types on the fly) load the created types into execution context and use them in your generated output.

Papacy

Answer 8

I think most of the things are already answered - as Slace said, you need some class that would be returned from the Select method. Once you have the class, you can use the System.Linq.Expressions.NewExpression method to create the expression.

If you really want to do this, you can generate class at runtime too. It's a bit more work, because it cannot be done using LINQ Expression trees, but it's possible. You can use System.Reflection.Emit namespace to do that - I just did a quick search and here is an article that explains this:

• Introduction to Creating Dynamic Types with Reflection.Emit

Syllabogram

Answer 9

You could use the Dynamic Expression API which allows you to dynamically build your select statement like this:

 Select("new(<property1>,<property2>,...)");

You need the Dynamics.cs file from the LINQ and language samples for Visual Studio for this to work, both are linked at the bottom of this page. You can also see a working example showing this in action on at the same URL.

International

I believe that will only work with LINQ to SQL, not another LINQ provider though – Sextillion

I believe the framework only works with IQueryable, not with IEnumerable. – International

i tryed u r code by it is giving error how to implement the above code in entity framework using datacontext? – Skipton