Using ternary conditional operator or expression in Entity Framework

Question

I have an entity framework query that I inherited which includes several Sums, cutdown example:-

from c in db.Clients
where {where clauses}
select new
{
ClientID = c.ClientID,
ClientName = c.ClientName,
CurrentBalance = c.ClientTransactions.Select(ct => ct.Amount)
                .DefaultIfEmpty(0m).Sum(),
}).ToList();

As the number of clients and the number of transactions has grown, this query has obviously become slower and slower.

Ideally I'd want to store balances rather than calculate them every time, but currently the system doesn't do that, and it would be a very large change to implement, so for now I'm just attempting a band-aid fix.

The fix I'm attempting to implement is to simply not do the Sum calculations (there are several, example above just has one) for people that aren't interested in them.

My first attempt was simply to use ternary conditional operators to determine whether or not to do the calculation:-

from c in db.Clients
where {where clauses}
select new
{
ClientID = c.ClientID,
ClientName = c.ClientName,
CurrentBalance = ClientSearchExcludeCurrentBalance ? 0m : 
                 c.ClientTransactions.Select(ct => ct.fAmount).DefaultIfEmpty(0m).Sum(),
}).ToList();

The problem with this, it turns out, is that regardless of the value of the condition (ClientSearchExcludeCurrentBalance) both sides are still calculated, and then the ternary decides which one to use. So even setting the condition to false, the Sum still gets processed and the query takes too long.

Commenting out the sum, as below...

from c in db.Clients
where {where clauses}
select new
{
ClientID = c.ClientID,
ClientName = c.ClientName,
CurrentBalance = ClientSearchExcludeCurrentBalance ? 0m : 0m,
                 //c.ClientTransactions.Select(ct => ct.fAmount).DefaultIfEmpty(0m).Sum(),
}).ToList();

... is now nice and fast, so the ternary is definitely running it even when it's not used.

So, with that idea out the window, I tried using an expression instead:-

Expression<Func<Client, Decimal>> currentBalance = c => 0m;
if (!ClientSearchExcludeCurrentBalance)
{
    currentBalance = c => c.ClientTransactions
                          .Select(ct => ct.Amount).DefaultIfEmpty(0m).Sum();
}

from c in db.Clients
where {where clauses}
select new
{
ClientID = c.ClientID,
ClientName = c.ClientName,
CurrentBalance = currentBalance.Invoke(c),
}).ToList();

This fell over with an unknown expression error:-

LINQ to Entities does not recognize the method 'System.Decimal Invoke[Client,Decimal](System.Linq.Expressions.Expression`1[System.Func`2[RPM.DAO.UI.Client,System.Decimal]], RPM.DAO.UI.Client)' method, and this method cannot be translated into a store expression

I also tried using Expand()

CurrentBalance = currentBalance.Expand().Invoke(c)

but still got the unknown expression error.

Just to see, I tried it with defaulting the Sum values to 0m, and then in the loop that assigns the results to the DTO Collection doing the sum there if needed

foreach (var client in Clients) 
{
    if (!ClientSearchExcludeCurrentBalance) {
        var c = db.Clients.FirstOrDefault(cl => cl.ClientID == client.ClientID);
        client.CurrentBalance = c.ClientTransactions.Select(ct => ct.fAmount)
                                .DefaultIfEmpty(0m).Sum();
    }
}

This works, in that it only does the sum if told to, but doing it outside the main select means the entire query now takes twice as long as it used to, so is clearly not viable.

So, my questions are:-

Does anyone know if it's possible to make Entity Framework only run the parts of a ternary conditional operator that will be used?

Does anyone know if it's possible to use an Expression to return a value in Entity Framework?

Or, alternatively, how to add an IF statement into an Entity Framework query?

For (non-working) example:-

from c in db.Clients
where {where clauses}
select new
{
ClientID = c.ClientID,
ClientName = c.ClientName,
CurrentBalance = if (ClientSearchExcludeCurrentBalance)
                     return 0m;
                 else 
                     return c.ClientTransactions.Select(tf => tf.fAmount)
                           .DefaultIfEmpty(0m).Sum(),
}).ToList();

Thanks!

Edit:

I tried Barr J's solution:-

from c in db.Clients
let currentBalance = ClientSearchExcludeCurrentBalance ? 0m : 
                     c.ClientTransactions.Select(ct => ct.Amount).DefaultIfEmpty(0m).Sum()
where {where clauses}
select new
{
ClientID = c.ClientID,
ClientName = c.ClientName,
CurrentBalance = currentBalance
}).ToList();

I get a null reference exception:

System.NullReferenceException: 'Object reference not set to an instance of an object.'

Edit #2: The cut-down version above doesn't give the null exception error, but the full version (with identical code) does... weird!

Anyway, with the working cut-down version above I tried it with the setting set to true and fall, and both took the same time, so it still does the Sum evaluation either way

Bradleybradly

Won't change the query shape but will prevent unnecessary data reads: `c.ClientTransactions.Where(ct => ClientSearchExcludeCurrentBalance).Select(ct => ...) – Eunaeunice

Answer 1

Linq will evaluate the operand from both sides regardless of the ternary operator, because it is being evaluated at run-time.

You will have to evaluate the operands outside of your linq statement and then use it.

for example:

var tst = from p in products join i in info on p.id equals i.pid

let status = p.type = "home" ? homestatus.Select(s=>s.status) :
             p.type = "offshore" ? offshorestatus.Select(s=>s.status) :
             p.type = "internal" ? internalestatus.Select(s=>s.status) : null
select new {
name = p.name,
status = status != null ? status.StatusText : string.Empty;
}

or:

var tst = from p in products join i in info on p.id equals i.pid

let status = (p.type = "home" ? homestatus.Select(s=>s.status.StatusText) :
             p.type = "offshore" ? offshorestatus.Select(s=>s.status.StatusText) :
             p.type = "internal" ? internalestatus.Select(s=>s.status.StatusText) : null) ?? string.Empty
select new {
name = p.name,
status = status;
}

Schwann

Answer 2

first of all: @Barr 's answer ist not correct. Not the evaluation at runtime is the problem (in the end, it is not evaluated at all for Linq To Entities!), but what the underlying provider of Linq2Entities tries to do:

Run though the whole expresion tree and build some valid SQL out of it. And of course, Find a SQL equivalent of "Invoke". Well there is nothing it can use so it throws the exception LINQ to Entities does not recognize the method

You have to avoid everything within such linq2entity statements that MUST be evaluated at runtime. E.g. access to DateTimeOffset.Now will also not work.

Currently I am not able to test your query so I can not tell you why the ternary operator does not work as expected. It may depend on how the SQL looks like.

I can give you two advices:

1. take a look at the query outcome. To do this, install SQL profiler (distributed with SQL Server installation), debug into your application until your linq2entities statement is executed. Hopefully you know that this will not happen unless you call ToList(), Any(), First() or something else onto the query. If you have no profiler, you should also be able to store the whole linq query in a variable (without calling toList()) and call ToString() onto it. This should you also give the query.

2. Have you thought about checking the execution plan of the query? This sounds like a missing index on ClientIdof the table Transaction. Maybe you can provide us with the SQL statement and/or the execution plan, so we will be able to help you more.

3. Additional hint: After retriving the query, you can execute it in SQL Management Studio. Please let you show the real execution plan. If you do this and there are some indices missing and SQL Server detects this missing index, it will suggest you what you can do to speed up the query.

Polygyny