Combining expressions c#

Question

I need to concatenate two expressions (with or statement)

My code:

var items = new List<Item>
{
    new Item { Color = "Black", Categories = new List<string> { "cat1", "cat2" } },
    new Item { Color = "Red", Categories = new List<string> { "cat3" } },
    new Item { Color = "White", Categories = new List<string> { "cat1" } }
};

var categories = new List<string> { "cat2", "cat3" };

Expression<Func<Item, bool>> func1 = (x1) => x1.Color == "Black";
Expression<Func<Item, bool>> func2 = (x2) => x2.Categories.Any(y => categories.Where(z => z == y).Any());
Expression<Func<Item, bool>> fullExpression = Expression.Lambda<Func<Item, bool>>(
        Expression.Or(func1.Body, func2.Body), func1.Parameters.Single());

var result = items.AsQueryable().Where(fullExpression);
// result should be like this
// items.Where(x => (x.Color == "Black") || x.Categories.Any(y => categories.Where(z => z == y).Any()))

I get run-time error variable 'x2' of type 'Item' referenced from scope '', but it is not defined'

I also was trying to build an expression with ExpressionVisitor.

Here is ExpressionVisitor:

internal class ParameterReplacer : ExpressionVisitor
{
    private readonly ParameterExpression _parameter;

    internal ParameterReplacer(ParameterExpression parameter)
    {
        _parameter = parameter;
    }

    protected override Expression VisitParameter(ParameterExpression node)
    {
        return base.VisitParameter(_parameter);
    }
}

Here how I use it in code:

Expression<Func<Item, bool>> func1 = (x1) => x1.Color == "Black";
Expression<Func<Item, bool>> func2 = (x2) => x2.Categories.Any(y => categories.Select(z => z == y).Any());
var paramExpr = Expression.Parameter(typeof(Item));
var exprBody = Expression.Or(func1.Body, func2.Body);
exprBody = (BinaryExpression)new ParameterReplacer(paramExpr).Visit(exprBody);
var finalExpr = Expression.Lambda<Func<Item, bool>>(exprBody, paramExpr);
var result = items.AsQueryable().Where(finalExpr);

In this case during creating ParameterReplacer I'm getting error

System.InvalidOperationException: 'The operands for operator 'Equal' do not match the parameters of method 'op_Equality'.'

What did I do wrong?

Poole

Please provide an minimal reproducible example – Cotto

@Cotto , the problem occurs when i'm trying to build fullExpression. What kind of details could I provide more? – Poole

It is exactly what it says. You referencing some unknown variable and not defining it in it's signatue. In your case it is your request variable. Second expression should be like: (x2, request) => ..... – Blayze

@Poole a full example someone can copy and run, in your case you should either simplify the example to the minimum that doesn't work (remove references to Item and request which are your own code) or otherwise provide their implementation if you can't repro the issue without them – Jaret

Question has been edited – Poole

I have added an answer. Let me know if you need any further help, I'm experienced in expression analysis and rewriting. – Faille

Answer 1

Ian Newson is entirely right but if you want code, here you go :)

Using these two classes you can combine the two predicates. I didn't come up with it but improved/adjusted it a bit and made it use the type Predicate instead of Func along with some newer language features (the original was quite old, sadly I don't remember where I found it).

internal class SubstExpressionVisitor : ExpressionVisitor
{
    private readonly Dictionary<Expression, Expression> _subst = new Dictionary<Expression, Expression>();

    protected override Expression VisitParameter(ParameterExpression node)
    {
        if (_subst.TryGetValue(node, out Expression newValue))
        {
            return newValue;
        }

        return node;
    }

    public Expression this[Expression original]
    {
        get => _subst[original];
        set => _subst[original] = value;
    }
}

public static class PredicateBuilder
{
    // you don't seem to need this but it's included for completeness sake
    public static Expression<Predicate<T>> And<T>(this Expression<Predicate<T>> a, Expression<Predicate<T>> b)
    {
        if (a == null)
            throw new ArgumentNullException(nameof(a));

        if (b == null)
            throw new ArgumentNullException(nameof(b));

        ParameterExpression p = a.Parameters[0];

        SubstExpressionVisitor visitor = new SubstExpressionVisitor();
        visitor[b.Parameters[0]] = p;

        Expression body = Expression.AndAlso(a.Body, visitor.Visit(b.Body));
        return Expression.Lambda<Predicate<T>>(body, p);
    }

    public static Expression<Predicate<T>> Or<T>(this Expression<Predicate<T>> a, Expression<Predicate<T>> b)
    {
        if (a == null)
            throw new ArgumentNullException(nameof(a));

        if (b == null)
            throw new ArgumentNullException(nameof(b));

        ParameterExpression p = a.Parameters[0];

        SubstExpressionVisitor visitor = new SubstExpressionVisitor();
        visitor[b.Parameters[0]] = p;

        Expression body = Expression.OrElse(a.Body, visitor.Visit(b.Body));
        return Expression.Lambda<Predicate<T>>(body, p);
    }
}

You can use it like this:

Expression<Predicate<Item>> func1 = (x1) => x1.Color == "Black";
Expression<Predicate<Item>> func2 = (x2) => x2.Categories.Any(y => categories.Select(z => z == y).Any());

Expression<Predicate<Item>> finalExpr = func1.Or(func2);

You might want to keep in mind that my Or is using OrElse internally so the second expression won't be evaluated if the one before is evaluated to true (OrElse is like ||, not |). The same goes for And with AndAlso (AndAlso is like &&, not &).
Another thing to note is that you can easily replace Predicate<T> with Func<T, bool> if you have to use Func for some reason :)

Sapling

I just realised, I don't think your statement about OrElse is true in this context. In .net parameters to method calls are all evaluated before the method is called. – Faille

Parameters yes (since they're also used in the first predicate). But the second expression won't be evaluated if the first one evaluates to true. OrElse is || while OR is |. – Sapling

Answer 2

This is because your two expressions (func1 and func2) reference two different ParameterExpressions. Just because they're of the same type doesn't mean they're the same.

They need to be the exact same ParameterExpression instance for this to work. For that you can leverage an expression rewriter to modify one of the expressions: https://learn.microsoft.com/en-us/dotnet/api/system.linq.expressions.expressionvisitor?view=netframework-4.8

I think you should be able to use a library like predicate builder to do the same thing in a simpler way though:

https://www.nuget.org/packages/PredicateBuilder/

EDIT:

Your code is on the right lines, but make the following changes:

For the visitor:

internal class ParameterReplacer : ExpressionVisitor
{
    private readonly ParameterExpression _parameter;

    internal ParameterReplacer(ParameterExpression parameter)
    {
        _parameter = parameter;
    }

    protected override Expression VisitParameter(ParameterExpression node)
    {
        return node;
    }
}

For the execution bit:

        Expression<Func<Item, bool>> func1 = (x1) => x1.Color == "Black";
        Expression<Func<Item, bool>> func2 = (x2) => x2.Categories.Any(y => categories.Select(z => z == y).Any());
        var paramExpr = func1.Parameters.Single();
        var expr2 = new ParameterReplacer(paramExpr).Visit(func1);
        var exprBody = Expression.Or(func1.Body, ((LambdaExpression)expr2).Body);

        var finalExpr = Expression.Lambda<Func<Item, bool>>(exprBody, paramExpr);

        var result = items.AsQueryable().Where(finalExpr)
            .ToList();

Faille

I've tried to apply ExpressionVisitor. Could you please recheck my answer, i've updated it. – Poole

Since you understand the expression stuff, what do you think of my solution as a more generic approach (aimed to be reusable)? – Sapling

@Sapling It looks good but limited to the ParameterExpression problem. Ultimately this kind of problem has been solved years ago so there's little reason to write your own solution for it. – Faille

What would be the alternative? Use that nuget? Why would you do that when you can just pop in these few lines in your project? Are there any things not covered by this (except that it's only aimed at Predicates of course)? I'm still learning about the expression-trees so any new insight is very welcome :) – Sapling

@Sapling For this one problem there's no issue. But ultimately if you're writing a full featured LINQ to something provider you'll want something that handles almost every NodeType. PredicateBuilder does that. If you move to the next level and you want to build your own expression interpreter you'll probably want some kind of intermediate model which can include your own meta data, because LINQ as a concept does not cater for that. – Faille

@IanNewson But in most cases you don't want to write your own linq provider, because you will probably not be able to implement it correctly.it's Always better to solve a simple issue you are having and not waste time on problems that you won't encounter. – Sick

@IanNewson Also PredicateBuilder is no longer maintained and would not recommend using it for projects linqkit github.com/scottksmith95/LINQKit is a good alternative. – Sick

@IanNewson , now it works without exceptions, but the result isn't I'd like to get. Only first func is applied.(where color is black). But I'd like to get back 2 items back (where color is black and where items have categories 2 and 3 ). Probably my question wasn't clear enough... so is it possible to do so? – Poole

@Poole Are you sure the conditions themselfs are correct? x2.Categories.Any(y => categories.Select(z => z == y).Any()) seems quite weird to me. You convert the categories to bools but don't check if the bools are true, instead you check if there are any bools. Do you just want to see if any category of x2 is present in categories? In that case you'd probably want x2.Categories.Intersect(categories).Any() (see Enumerable.Intersect). – Sapling

@Sapling Actually you're right. I corrected my answer in morning. Btw, you solution fit me well, Thank you a lot! P.S. Do you happen to know will it expression work on DB-side if using MongoDb driver? – Poole

@IanNewson dear Ian, Thank you for your assistance as well, it was helpful. – Poole