Using a partial class property inside LINQ statement
Asked Answered
H

6

23

I am trying to figure out the best way to do what I thought would be easy. I have a database model called Line that represents a line in an invoice.

It looks roughly like so:

public partial class Line 
{
    public Int32 Id { get; set; }
    public Invoice Invoice { get; set; }
    public String Name { get; set; }
    public String Description { get; set; }
    public Decimal Price { get; set; }
    public Int32 Quantity { get; set; }
}

This class is generated from the db model.
I have another class that adds one more property:

public partial class Line
{
    public Decimal Total
    {
        get
        {
            return this.Price * this.Quantity
        }
    }
}

Now, from my customer controller I want to do something like this:

var invoices = ( from c in _repository.Customers
                         where c.Id == id
                         from i in c.Invoices
                         select new InvoiceIndex
                         {
                             Id = i.Id,
                             CustomerName = i.Customer.Name,
                             Attention = i.Attention,
                             Total = i.Lines.Sum( l => l.Total ),
                             Posted = i.Created,
                             Salesman = i.Salesman.Name
                         }
        )

But I can't thanks to the infamous

The specified type member 'Total' is not supported in LINQ to Entities. Only initializers, entity members, and entity navigation properties are supported.

What is the best way to refactor this so that it works?

I have tried LinqKit, i.Lines.AsEnumerable(), and putting i.Lines in my InvoiceIndex model and having it calculate the sum for the view.

That last solution 'works' but I cannot sort on that data. What I want to be able to do in the end is

var invoices = ( from c in _repository.Customers
                         ...
        ).OrderBy( i => i.Total )

Also I want to page my data, so I do not want to waste time converting the entire c.Invoices to a list with .AsEnumerable()

Bounty

I know this must be a somewhat big problem for some people. After hours of scouring the internet I have come to the conclusion that no happy conclusion has been made. Yet I believe this must be a fairly common roadblock for those who are trying to do paging and sorting with ASP MVC. I understand that the property can not be mapped to sql and therefore you cannot sort on it before paging, but what I am looking for is a way to get my desired result.

Requirements for a perfect solution:

  • DRY, meaning my total calculations would exist in 1 place
  • Support both sorting and paging, and in that order
  • Not pull the whole table of data into memory with .AsEnumerable or .AsArray

What I would be really happy to find is a way to specify the Linq to entities SQL in my extended partial class. But I have been told this is not possible. Note that a solution does not need to directly use the Total property. Calling that property from IQueryable is not supported at all. I am looking for a way to achieve the same result via a different method, yet equally simple and orthogonal.

The winner of the bounty will be the solution with the highest votes at the end, unless someone posts a perfect solution :)

Ignore below unless until you read the answer(s):

{1} Using Jacek's solution I took it one step further and made the properties invokable using LinqKit. This way even the .AsQueryable().Sum() is enclosed in our partial classes. Here is some examples of what I am doing now:

public partial class Line
{
    public static Expression<Func<Line, Decimal>> Total
    {
        get
        {
            return l => l.Price * l.Quantity;
        }
    }
}

public partial class Invoice
{
    public static Expression<Func<Invoice, Decimal>> Total
    {
        get
        {
            return i => i.Lines.Count > 0 ? i.Lines.AsQueryable().Sum( Line.Total ) : 0;
        }
    }
}

public partial class Customer
{
    public static Expression<Func<Customer, Decimal>> Balance
    {
        get
        {
            return c => c.Invoices.Count > 0 ? c.Invoices.AsQueryable().Sum( Invoice.Total ) : 0;
        }
    }
}

First trick was the .Count checks. Those are needed because I guess you cannot call .AsQueryable on an empty set. You get an error about Null materialization.

With these 3 partial classes laid out you can now do tricks like

var customers = ( from c in _repository.Customers.AsExpandable()
                           select new CustomerIndex
                           {
                               Id = c.Id,
                               Name = c.Name,
                               Employee = c.Employee,
                               Balance = Customer.Balance.Invoke( c )
                           }
                    ).OrderBy( c => c.Balance ).ToPagedList( page - 1, PageSize );

var invoices = ( from i in _repository.Invoices.AsExpandable()
                         where i.CustomerId == Id 
                         select new InvoiceIndex
                        {
                            Id = i.Id,
                            Attention = i.Attention,
                            Memo = i.Memo,
                            Posted = i.Created,
                            CustomerName = i.Customer.Name,
                            Salesman = i.Salesman.Name,
                            Total = Invoice.Total.Invoke( i )
                        } )
                        .OrderBy( i => i.Total ).ToPagedList( page - 1, PageSize );

Very cool.

There is a catch, LinqKit does not support the invocation of properties, you will get an error about trying to cast PropertyExpression to LambaExpression. There are 2 ways around this. Firstly is to pull the expression yourself like so

var tmpBalance = Customer.Balance;
var customers = ( from c in _repository.Customers.AsExpandable()
                           select new CustomerIndex
                           {
                               Id = c.Id,
                               Name = c.Name,
                               Employee = c.Employee,
                               Balance = tmpBalance.Invoke( c )
                           }
                    ).OrderBy( c => c.Balance ).ToPagedList( page - 1, PageSize );

which I thought was kind of silly. So I modified LinqKit to pull out the get{} value when it encounters a property. The way it operates on the expression is similar to reflection, so its not like the compiler is going to resolve Customer.Balance for us. There is a 3 line change I made to TransformExpr in ExpressionExpander.cs. Its probably not the safest code and might break other things, but it works for now and I have notified the author about the deficiency.

Expression TransformExpr (MemberExpression input)
{
        if( input.Member is System.Reflection.PropertyInfo )
        {
            return Visit( (Expression)( (System.Reflection.PropertyInfo)input.Member ).GetValue( null, null ) );
        }
        // Collapse captured outer variables
        if( input == null

In fact I pretty much guarantee that this code will break some things, but it works at the moment and that is good enough. :)

Hughhughes answered 29/7, 2011 at 22:8 Comment(1)
I think that your last update really should be its own answer.Strickland
P
23

There is another way, which is a bit more complex, but gives you the ability to encapsulate this logic.

public partial class Line
{
    public static Expression<Func<Line,Decimal>> TotalExpression
    {
        get
        {
            return l => l.Price * l.Quantity
        }
    }
}

Then rewrite the query to

var invoices = ( from c in _repository.Customers
                     where c.Id == id
                     from i in c.Invoices
                     select new InvoiceIndex
                     {
                         Id = i.Id,
                         CustomerName = i.Customer.Name,
                         Attention = i.Attention,
                         Total = i.Lines.AsQueryable().Sum(Line.TotalExpression),
                         Posted = i.Created,
                         Salesman = i.Salesman.Name
                     }
               )

It worked for me, performs queries server-side and complies with the DRY rule.

Pollaiuolo answered 3/8, 2011 at 13:45 Comment(4)
+1: Not bad! I expected using AsQueryable wouldn't be possible in a LINQ to Entities projection. But it indeed works (just tested it). Will keep this nice "trick" in mind.Heteronym
I really like this solution. Currently I am wrestling with LinqKit to try and make the property invokable. Which would turn the line into Total = Invoice.Total.Invoke( i ) :D Currently LinqKit does not seem to support invoking properties, but I think its doableHughhughes
Ok so long story short, I got linqkit working and this is a pretty damn good solution :) See my edits under {1} in the question for some code.Hughhughes
Yes, good work. I did similar things while i struggled to make dynamic Linq queries in DevExpress XPO. Of course, this will break if you use the property's property and so on. Still, it gives loads of satisfaction :)Thrombocyte
L
5

Your extra property is simply a calculation of the data from the model, but the property is not something EF can naturally translate. SQL is perfectly capable of performing this same calculation, and EF can translate the calculation. Use it in place of the property if you need it in your query.

Total = i.Lines.Sum( l => l.Price * l.Quantity)
Labefaction answered 30/7, 2011 at 2:56 Comment(3)
This works, but lets say I do this in 5 different places and a month later I need to add a tax calculation or shipping charges, or an extra charge based on the item's weight.. I would like to keep the 'Total' calculation encapsulated in the Line model.Hughhughes
I wonder if its possible to inform EF about the type calculation in my code... something that would say "Output [ Price * Quantity ] when building the sql command for this value" That would be niceHughhughes
@Charles: No it is not possible to map custom calculation to property. Your linq-to-entities queries can contain only mapped (persisted) properties. Custom calculation can be wrapped in model defined functions but it is more complex.Shick
A
3

Try something like this:

var invoices =
    (from c in _repository.Customers
    where c.Id == id
    from i in c.Invoices
    select new 
    {
        Id = i.Id,
        CustomerName = i.Customer.Name,
        Attention = i.Attention,
        Lines = i.Lines,
        Posted = i.Created,
        Salesman = i.Salesman.Name
    })
    .ToArray()
    .Select (i =>  new InvoiceIndex
    {
        Id = i.Id,
        CustomerName = i.CustomerName,
        Attention = i.Attention,
        Total = i.Lines.Sum(l => l.Total),
        Posted = i.Posted,
        Salesman = i.Salesman
    });

Basically this queries the database for the records you need, brings them into memory and then does the summing once the data is there.

Attractant answered 30/7, 2011 at 2:45 Comment(5)
The ToArray can actually be skipped. Instead use AsEnumerable() which hides the IQueryable from that point on which helps to maintain the streaming nature instead of buffering/allocating array storage.Lukash
Now this one is interesting. I did not think of using an anonymous type like this. I am currently using something similar with AsEnumerable, but from what I understand using ToArray and AsEnumerable causes the SQL chain to end and the code will go fetch ALL the invoice data before allowing me to do my total calc. This is undesirable because I am paginating results and I only need 10 or so invoices in 1 go. I may have to give up on that requirement though, I just can't think of a way that it can be possible except to follow Anthony's solutionHughhughes
@Hughhughes - could you not do your pagination before the ToArray/AsEnumerable call?Attractant
But page 4 sorted by Id will be different data than page 4 sorted by Total. I could switch to paging-then-sorting, but it would be kind of weird for alphabetical sorts.Hughhughes
I think what I may end up doing is extending my sorter so when it sorts on table data it uses a simple .OrderBy, but if it sorts on a calculated field it uses .AsEnumerable first. That way my users would only take the performance hit when they are sorting on a 'bad' field.Hughhughes
C
3

I think easiest way to resolve this problem is using DelegateDecompiler.EntityFramework (make by Alexander Zaytsev)

It's a library which is able to decompile a delegate or a method body to its lambda representation.


Explain:

Asume we have a class with a computed property

class Employee
{
    [Computed]
    public string FullName
    {
        get { return FirstName + " " + LastName; }
    }

    public string LastName { get; set; }

    public string FirstName { get; set; }
}

And you are going to query employees by their full names

var employees = (from employee in db.Employees
                 where employee.FullName == "Test User"
                 select employee).Decompile().ToList();

When you call .Decompile method it decompiles your computed properties to their underlying representation and the query will become simmilar to the following query

var employees = (from employee in db.Employees
                 where (employee.FirstName + " " + employee.LastName)  == "Test User"
                 select employee).ToList();

If your class doesn't have a [Computed] attribute, you can use the .Computed() extension method..

var employees = (from employee in db.Employees
                 where employee.FullName.Computed() == "Test User"
                 select employee).ToList();

Also, you can call methods that return a single item (Any, Count, First, Single, etc) as well as other methods in identical way like this:

bool exists = db.Employees.Decompile().Any(employee => employee.FullName == "Test User");

Again, the FullName property will be decompiled:

bool exists = db.Employees.Any(employee => (employee.FirstName + " " + employee.LastName) == "Test User");

Async Support with EntityFramework

The DelegateDecompiler.EntityFramework package provides DecompileAsync extension method which adds support for EF's Async operations.


Additionally

You can find 8 way to mix some property values together in EF here:

Computed Properties and Entity Framework. (written by Dave Glick)

Charlatanry answered 30/11, 2016 at 5:33 Comment(2)
Its nice that there is a true package to support this now!Hughhughes
@Charles, Yes that is a nice and useful package. Also, I added a reference to my answer about other ways.Charlatanry
J
1

While not an answer to your subject, it may be an answer to your problem:

Since you seem willing to modify the database, why not create a new View in the database that is basically

create view TotalledLine as
select *, Total = (Price * Quantity)
from LineTable;

and then change your data model to use TotalledLine instead of LineTable?

After all, if your application needs Total, it isn't a stretch to think others might, and centralizing that type of calculation into the database is one reason for using a database.

Jotter answered 4/8, 2011 at 23:8 Comment(3)
I did think about a view, but I do not think the entity framework can work with them (very easily) and they are not designable in the db designer.Hughhughes
The query and view designer is used to maintain views, and EF works with views pretty well. What issues concern you?Jotter
the calculation is also simple enough you can trigger an update, or implement a 'close order' workflow that finalizes a few values. (with or without any dbms support.) personally i try to remain as db-agnostic as possible these days, in the last 10 years i've seen more back-ends swap than i like to admit.. and dbms artifacts have been some of the most expensive artifacts to deal withMucosa
H
0

Just to add some of my own thoughts here. Because I am such a perfectionist I do not want to pull the entire contents of the table into memory just so I can do the calculation then sort then page. So somehow the table needs to know what the total is.

What I was thinking was to create a new field in the line table called 'CalcTotal' that contains the calculated total of the line. This value would be set every time the line is changed, based on the value of .Total

This has 2 advantages. Firstly, I can change the query to this:

var invoices = ( from c in _repository.Customers
                     where c.Id == id
                     from i in c.Invoices
                     select new InvoiceIndex
                     {
                         Id = i.Id,
                         CustomerName = i.Customer.Name,
                         Attention = i.Attention,
                         Total = i.Lines.Sum( l => l.CalcTotal ),
                         Posted = i.Created,
                         Salesman = i.Salesman.Name
                     }
    )

Sorting and paging will work because its a db field. And I can do a check ( line.CalcTotal == line.Total ) in my controller to detect any tom foolery. It does cause a little bit of overhead in my repository, namely when I go to save or create a new line I would have to add

line.CalcTotal = line.Total

but I think it may be worth it.

Why go to the trouble of having 2 properties that have the same data?

Well its true, I could put

line.CalcTotal = line.Quantity * line.Price;

in my repository. This would not be very orthogonal though, and if some change to line totals needed to happen, it would make much more sense to edit the partial Line class than the Line repository.

Hughhughes answered 1/8, 2011 at 6:20 Comment(2)
If you are prepared to change the database why not just create a computed field in the database and set it to storegenerated?Platy
I was looking into computed fields but I could not find a perfect solution. The computed column would need to be the sum of another computed column in a related table + any extra costs. Also I do not believe there is a way to designate the formulas in the EF db designer. But if someone posts a cool solution with them I wouldn't ignore it ;)Hughhughes

© 2022 - 2024 — McMap. All rights reserved.