EF Core 2.0.0 Query Filter is Caching TenantId (Updated for 2.0.1+)

Question

I'm building a multi-tenant application, and am running into difficulties with what I think is EF Core caching the tenant id across requests. The only thing that seems to help is constantly rebuilding the application as I sign in and out of tenants.

I thought it may have something to do with the IHttpContextAccessor instance being a singleton, but it can't be scoped, and when I sign in and out without rebuilding I can see the tenant's name change at the top of the page, so it's not the issue.

The only other thing I can think of is that EF Core is doing some sort of query caching. I'm not sure why it would be considering that it's a scoped instance and it should be getting rebuild on every request, unless I'm wrong, which I probably am. I was hoping it would behave like a scoped instance so I could simply inject the tenant id at model build time on each instance.

I'd really appreciate it if someone could point me in the right direction. Here's my current code:

TenantProvider.cs

public sealed class TenantProvider :
    ITenantProvider {
    private readonly IHttpContextAccessor _accessor;

    public TenantProvider(
        IHttpContextAccessor accessor) {
        _accessor = accessor;
    }

    public int GetId() {
        return _accessor.HttpContext.User.GetTenantId();
    }
}

...which is injected into TenantEntityConfigurationBase.cs where I use it to setup a global query filter.

internal abstract class TenantEntityConfigurationBase<TEntity, TKey> :
    EntityConfigurationBase<TEntity, TKey>
    where TEntity : TenantEntityBase<TKey>
    where TKey : IEquatable<TKey> {
    protected readonly ITenantProvider TenantProvider;

    protected TenantEntityConfigurationBase(
        string table,
        string schema,
        ITenantProvider tenantProvider) :
        base(table, schema) {
        TenantProvider = tenantProvider;
    }

    protected override void ConfigureFilters(
        EntityTypeBuilder<TEntity> builder) {
        base.ConfigureFilters(builder);

        builder.HasQueryFilter(
            e => e.TenantId == TenantProvider.GetId());
    }

    protected override void ConfigureRelationships(
        EntityTypeBuilder<TEntity> builder) {
        base.ConfigureRelationships(builder);

        builder.HasOne(
            t => t.Tenant).WithMany().HasForeignKey(
            k => k.TenantId);
    }
}

...which is then inherited by all other tenant entity configurations. Unfortunately it doesn't seem to work as I had planned.

I have verified that the tenant id being returned by the user principal is changing depending on what tenant user is logged in, so that's not the issue. Thanks in advance for any help!

Update

For a solution when using EF Core 2.0.1+, look at the not-accepted answer from me.

Update 2

Also look at Ivan's update for 2.0.1+, it proxies in the filter expression from the DbContext which restores the ability to define it once in a base configuration class. Both solutions have their pros and cons. I've opted for Ivan's again because I just want to leverage my base configurations as much as possible.

Grayish

Why can't IHttpContextAccessor be scoped/transient? Might be worth showing relevant elements of your dependency injection config. – Blancmange

@CalC: Because it is a singleton (basically a kind of a factory) and calling it's HttpContext property retrieves the http context of the current request – Weinstein

Is TenantProvider scoped or singleton? And your DbContext instances singletons too? Too me it seems something survives the request, often hint of lifetime issues with your IoC – Weinstein

@Weinstein TenantProvider and the DbContext are both scoped instances. Ivan's answer solved it because it was just the way EF works. – Grayish

Answer 1

Currently (as of EF Core 2.0.0) the dynamic global query filtering is quite limited. It works only if the dynamic part is provided by direct property of the target DbContext derived class (or one of its base DbContext derived classes). Exactly as in the Model-level query filters example from the documentation. Exactly that way - no method calls, no nested property accessors - just property of the context. It's sort of explained in the link:

Note the use of a DbContext instance level property: TenantId. Model-level filters will use the value from the correct context instance. i.e. the one that is executing the query.

To make it work in your scenario, you have to create a base class like this:

public abstract class TenantDbContext : DbContext
{
    protected ITenantProvider TenantProvider;
    internal int TenantId => TenantProvider.GetId();
}

derive your context class from it and somehow inject the TenantProvider instance into it. Then modify the TenantEntityConfigurationBase class to receive TenantDbContext:

internal abstract class TenantEntityConfigurationBase<TEntity, TKey> :
    EntityConfigurationBase<TEntity, TKey>
    where TEntity : TenantEntityBase<TKey>
    where TKey : IEquatable<TKey> {
    protected readonly TenantDbContext Context;

    protected TenantEntityConfigurationBase(
        string table,
        string schema,
        TenantDbContext context) :
        base(table, schema) {
        Context = context;
    }

    protected override void ConfigureFilters(
        EntityTypeBuilder<TEntity> builder) {
        base.ConfigureFilters(builder);

        builder.HasQueryFilter(
            e => e.TenantId == Context.TenantId);
    }

    protected override void ConfigureRelationships(
        EntityTypeBuilder<TEntity> builder) {
        base.ConfigureRelationships(builder);

        builder.HasOne(
            t => t.Tenant).WithMany().HasForeignKey(
            k => k.TenantId);
    }
}

and everything will work as expected. And remember, the Context variable type must be a DbContext derived class - replacing it with interface won't work.

Update for 2.0.1: As @Smit pointed out in the comments, v2.0.1 removed most of the limitations - now you can use methods and sub properties.

However, it introduced another requirement - the dynamic expression must be rooted at the DbContext.

This requirement breaks the above solution, since the expression root is TenantEntityConfigurationBase<TEntity, TKey> class, and it's not so easy to create such expression outside the DbContext due to lack of compile time support for generating constant expressions.

It could be solved with some low level expression manipulation methods, but the easier in your case would be to move the filter creation in generic instance method of the TenantDbContext and call it from the entity configuration class.

Here are the modifications:

TenantDbContext class:

internal Expression<Func<TEntity, bool>> CreateFilter<TEntity, TKey>()
    where TEntity : TenantEntityBase<TKey>
    where TKey : IEquatable<TKey>
{
    return e => e.TenantId == TenantId;
}

TenantEntityConfigurationBase<TEntity, TKey> class:

builder.HasQueryFilter(Context.CreateFilter<TEntity, TKey>());

Aneroid

Sure enough, changing it to your example made it work exactly as I wanted it to. Now, I understand how it works currently, but I don't understand why it is this way, it just feels much more complicated than it should/could be. Side question, is it possible to set a default tenant id for new entities being added without me having to explicitly do it prior to passing the new entity to EF? – Grayish

Interesting question, didn't think about that. Eventually utilizing custom ValueGenerator, but not sure how to get access to the TenantProvider. Or overriding SaveChanges and processing added entities. In both cases you'll need a base non generic entity class like TenantEntityBase (w/o TKey) to identify them and have access to TenantId property. – Aneroid

Lol, I just peeked at your profile and I see you're a fellow Bulgarian as well. Привет от щатите! – Grayish

" Exactly that way - no method calls, no nested property accessors - just property of the context." That limitation is true for 2.0.0. It was very restrictive as in example only. After looking at codebase for 2.0.1, we removed that restriction while fixing a different bug. All things mentioned is supported as long as it is rooted at context in the expression tree. – Trisaccharide

@Trisaccharide My answer is based on 2.0.0 (I didn;'t even know there is 2.0.1)It's hard to monitor the changes in your major builds, what to say about minor. Are there any human readable documentation explaining all limitations/bugs and what is fixed/included in every released build? Don't tell me that we have to go searching through GitHub issue tracker :) I usually as OP to specify the EFC version like 1.0, 1.1. 2.0, now have to ask for minor versions too :) – Aneroid

This could help out a bit github.com/aspnet/EntityFrameworkCore/releases/tag/2.0.1 As for limitations, some of them are mentioned in blogpost associated with release. But those are generally major limitations which had cost causing us to delay implementation. While this is also major limitation, it ended up there unintentionally. So till people started hitting it we did not have idea that its a limitation. But once we come to know, we improve it. – Trisaccharide

Further, generally limitations are not supposed to be removed in minor releases. (its feature, shouldn't be part of patch). We accidentally ended up doing while fixing another bug. – Trisaccharide

@Trisaccharide Got it. Thanks for your time commenting and explaining (and link). – Aneroid

@IvanStoev 1. Can it work with code first approach? 2. EntityConfigurationBase - cannot resolve. Is it class which I have to create by myself or is it coming from EF ? I have ITenantEntity which is implemented by every Tenant's model – Unipersonal

@MU 1. It is code first approach 2. EntityConfigurationBase is some class by the OP. You can simply ignore it. Also it's not necessary to use separate classes for configuration - you can simply put all the configuration code inside the OnModelCreating. For setting the filter for every ITenantEntity you could use similar loop as in the other question. But the filter should use TenantId property of the context, otherwise it won't be dynamic. Like example in docs.microsoft.com/en-us/ef/core/what-is-new/…, but for every entity implementing ITenantEntity. – Aneroid

Tried it: builder.Entity<ITenantEntity>().HasQueryFilter(pe => pe.TenantId== TenantId); but it's throwing error The entity type ITenantEntity provided for the argument 'clrType' must be a reference type. – Unipersonal

I have tried: foreach (var type in builder.Model.GetEntityTypes()) { if (typeof(ITenantEntity).IsAssignableFrom(type.ClrType)) builder.Entity(type.ClrType).HasQueryFilter(pe => pe.TenantId == _TenantId); } but then I have an error of "lamba expression not assignable to parameter type "System.Linq.Expressions.LambdaExpression" – Unipersonal

@MU Create a generic instance method inside you db context: private void SetTenantFilter<TEntity>(ModelBuilder modelBuilder) where TEntity : class, ITenantEntity { modelBuilder.Entity<TEntity>().HasQueryFilter(e => e.TenantId == this._TenantId); } Then call it using reflection inside the loop: foreach (var type in modelBuilder.Model.GetEntityTypes()) { if (typeof(ITenantEntity).IsAssignableFrom(type.ClrType)) GetType().GetMethod("SetTenantFilter", BindingFlags.Instance | BindingFlags.NonPublic).MakeGenericMethod(type.ClrType).Invoke(this, new[] { modelBuilder }); } – Aneroid

1. THANK YOU! It's working. Any reading which would help me to really understand what's going on there? I don't like just pasting the code. 2. Can you just point me the direction how should I proceed to secure saving/updates/deletions? – Unipersonal

@MU 1. Unfortunately there is not much in this area, so it's based on my personal experiments. I'm showing different approaches of setting the query filter for multiple entities here EF Core: Soft delete with shadow properties and query filters. 2. I have no clue in that direction, probably the same techniques used in EF6 like overriding SaveChanges and examining ChangeTracker.Entries() – Aneroid

Answer 2

Answer for 2.0.1+

So, the day I got it work, EF Core 2.0.1 was released. As soon as I updated, this solution came crashing down. After a very long thread over here, it turned out that it was really a fluke that it was working in 2.0.0.

Officially for 2.0.1 and beyond any query filters that depend on an outside value, like the tenant id in my case, must be defined in the OnModelCreating method and must reference a property on the DbContext. The reason is because on first run of the app or first call into EF all EntityTypeConfiguration classes are processed and their results are cached regardless of how many times the DbContext is instanced.

That's why defining the query filters in the OnModelCreating method works because it's a fresh instance and the filter lives and dies with it.

public class MyDbContext : DbContext {
    private readonly ITenantService _tenantService;

    private int TenantId => TenantService.GetId();

    public DbSet<User> Users { get; set; }

    public MyDbContext(
        DbContextOptions options,
        ITenantService tenantService) {
        _tenantService = tenantService;
    }

    protected override void OnModelCreating(
        ModelBuilder modelBuilder) {
        modelBuilder.Entity<User>().HasQueryFilter(
            u => u.TenantId == TenantId);
    }
}

Grayish

It has nothing to do with caching and context instances. It's pure expression trees related problem - see my update. – Aneroid

Answer 3

Update: Unfortunately this won't work as expected... I looked at the SQL log and the function in the lambda expression is not evaluated which will result in a full resultset being returned and then filtered on the client side.

I use the following pattern to be able to externally add filters without having a property on the context itself.

    public class QueryFilters
    {
        internal static IDictionary<Type, List<LambdaExpression>> Filters { get; set; } = new Dictionary<Type, List<LambdaExpression>>();

        public static void RegisterQueryFilter<T>(Expression<Func<T, bool>> expression)
        {
            List<LambdaExpression> list = null;
            if (Filters.TryGetValue(typeof(T), out list) == false)
            {
                list = new List<LambdaExpression>();
                Filters.Add(typeof(T), list);
            }

            list.Add(expression);
        }
    }

And in my context I add the query filters like so:

    public class MyDbContext : DbContext
    {
        protected override void OnModelCreating(ModelBuilder modelBuilder)
        {
            base.OnModelCreating(modelBuilder);

            foreach (var type in QueryFilters.Filters.Keys)
                foreach (var filter in QueryFilters.Filters[type])
                    modelBuilder.Entity(type).HasQueryFilter(filter);
        }
    }

And I register my query filters somewhere else (i.e. in some configuration code) like this:

    Func<User, bool> func = i => IncludeSoftDeletedEntities.DisableFilter;
    QueryFilters.RegisterQueryFilter<User>(i => func(i) || EF.Property<bool>(i, "IsDeleted") == false);

In this example I'm adding a soft-delete filter which can be disabled using the "global" IncludeSoftDeletedEntities.DisableFilter (which is actually powered by a scope mechanism).

The snags here are that EF.Property cannot be used outside the actual expression, so it needs to be where it is. Another thing to mention is that we need to encapsulate any logic in a Func to avoid it being "cached".

Hallsy