LazyCache: Regularly refresh cached items

Question

I am using LazyCache and want to have cache refreshed e.g. every hour, but ideally I want the first caller after the cache item expired do not wait for cache reloaded. I wrote the following

public async Task<List<KeyValuePair<string, string>>> GetCarriersAsync()
{

    var options = new MemoryCacheEntryOptions
    {
        AbsoluteExpirationRelativeToNow = new TimeSpan(1,0,0),// consider to config
    }.RegisterPostEvictionCallback(
         async  (key, value, reason, state) =>
        {
            await GetCarriersAsync();//will save to cache
            _logger.LogInformation("Carriers are reloaded: " );
        });
    Func<Task<List<KeyValuePair<string, string>>>> cacheableAsyncFunc = () => GetCarriersFromApi();
    var cachedCarriers = await _cache.GetOrAddAsync($"Carriers", cacheableAsyncFunc, options);

    return cachedCarriers;
}

However RegisterPostEvictionCallback is not called when cache item is expired, but only when the next request to the item occurred (and the caller need to wait for a lengthy operation).

The thread Expiration almost never happens on it's own in the background #248 explains that this is by design, and suggests workaround to specify CancellationTokenSource.CancelAfter(TimeSpan.FromHours(1)) instead of SetAbsoluteExpiration.

Unfortunately LazyCache.GetOrAddAsync doesn’t have CancellationToken as a parameter. What is the best way to trigger reload of cache on a scheduled time with minimal waiting time for the first user?

Adumbrate

Answer 1

I found the similar question In-Memory Caching with auto-regeneration on ASP.Net Core that suggested to call the AddExpirationToken(new CancellationChangeToken(new CancellationTokenSource(_options.ReferenceDataRefreshTimeSpan).Token).

I tried it, but didn't make it working. However the same answer had alternative(and recommended) option by using timer. I've created a class RefreshebleCache that I am using for different cachable options like the following:

   var refreshebleCache = new RefreshebleCache<MyCashableObjectType>(_cache, _logger);
   Task<MyCashableObjectType> CacheableAsyncFunc() => GetMyCashableObjectTypeFromApiAsync();
   var cachedResponse = await refreshebleCache.GetOrAddAsync("MyCashableObject", CacheableAsyncFunc,
                        _options.RefreshTimeSpan);

The RefreshebleCache implementation:

/// <summary>
    /// Based on https://mcmap.net/q/1075019/-in-memory-caching-with-auto-regeneration-on-asp-net-core
    /// </summary>
    /// <typeparam name="T"></typeparam>
    public class RefreshebleCache<T>
    {

        protected readonly IAppCache _cache;
        private readonly ILogger _logger;
        public bool LoadingBusy = false;
        private string _cacheKey;
        private TimeSpan _refreshTimeSpan;
        private Func<Task<T>> _functionToLoad;
        private Timer _timer;

        public RefreshebleCache(IAppCache cache, ILogger logger)
        {

            _cache = cache;
            _logger = logger;
        }

        public async Task<T>  GetOrAddAsync (string cacheKey , Func<Task<T>> functionToLoad, TimeSpan refreshTimeSpan)
        {
            _refreshTimeSpan= refreshTimeSpan;
            _functionToLoad = functionToLoad;
            _cacheKey = cacheKey;
            var timerCachedKey = "Timer_for_"+cacheKey;
            //if removed from cache, _timer could continue to work, creating redundant calls
            _timer =  _appCache.GetOrAdd(timerCachedKey, () => 
             CreateTimer(refreshTimeSpan), 
  SetMemoryCacheEntryOptions(CacheItemPriority.NeverRemove));
            var cachedValue = await LoadCacheEntryAsync();
            return  cachedValue;
        }
        private Timer CreateTimer(TimeSpan refreshTimeSpan)
        {
            Debug.WriteLine($"calling CreateTimer for {_cacheKey} refreshTimeSpan {refreshTimeSpan}"); //start first time in refreshTimeSpan
            return new Timer(TimerTickAsync, null, refreshTimeSpan, refreshTimeSpan);
        }

    
        private async void TimerTickAsync(object state)
        {
            if (LoadingBusy) return;
            try
            {
                LoadingBusy = true;
                Debug.WriteLine($"calling LoadCacheEntryAsync from TimerTickAsync for {_cacheKey}");
                var loadingTask = LoadCacheEntryAsync(true);
                await loadingTask;
            }
            catch(Exception e)
            {
                _logger.LogWarning($" {nameof(T)} for {_cacheKey} was not reloaded.    {e} ");
            }
            finally
            {
                LoadingBusy = false;
            }
        }
        private async Task<T> LoadCacheEntryAsync(bool update=false)
        {
            var cacheEntryOptions = SetMemoryCacheEntryOptions();

            Func<Task<T>> cacheableAsyncFunc = () => _functionToLoad();
            Debug.WriteLine($"called LoadCacheEntryAsync for {_cacheKey} update:{update}");
            T cachedValues = default(T);
            if (update)
            {
                cachedValues =await cacheableAsyncFunc();
                if (cachedValues != null)
                {
                    _cache.Add(_cacheKey, cachedValues, cacheEntryOptions);
                }

                //    _cache.Add(_cacheKey, cacheableAsyncFunc, cacheEntryOptions);
            }
            else
            {
                 cachedValues = await _cache.GetOrAddAsync(_cacheKey, cacheableAsyncFunc, cacheEntryOptions);
            }
            return cachedValues;
        }
        private MemoryCacheEntryOptions SetMemoryCacheEntryOptions(CacheItemPriority priority= CacheItemPriority.Normal)
       {
          var cacheEntryOptions = new MemoryCacheEntryOptions
          {
            Priority = priority 
          };
          return cacheEntryOptions;
        }

 }

}

Adumbrate

If you use a timer, you may not need to expire the entry with AbsoluteExpirationRelativeToNow. Otherwise you may reach cases where your cache will be empty. The timer is getting you fresh content anyway. – Neusatz

Answer 2

Auto refresh can now be achieved with LazyCache 2.1, using LazyCacheEntryOptions and ExpirationMode.ImmediateExpiration which are really just wrappers for time delayed cancellation tokens. You can see this being demonstrated in the following test taken from the LazyCache test suite:

        [Test]
        public async Task AutoRefresh()
        {
            var key = "someKey";
            var refreshInterval = TimeSpan.FromSeconds(1);
            var timesGenerated = 0;

            // this is the Func what we are caching 
            ComplexTestObject GetStuff()
            {
                timesGenerated++;
                return new ComplexTestObject();
            }

            // this sets up options that will recreate the entry on eviction
            MemoryCacheEntryOptions GetOptions()
            {
                var options = new LazyCacheEntryOptions()
                    .SetAbsoluteExpiration(refreshInterval, ExpirationMode.ImmediateExpiration);
                options.RegisterPostEvictionCallback((keyEvicted, value, reason, state) =>
                {
                    if (reason == EvictionReason.Expired  || reason == EvictionReason.TokenExpired)
                        sut.GetOrAdd(key, _ => GetStuff(), GetOptions());
                });
                return options;
            }

            // get from the cache every 2s
            for (var i = 0; i < 3; i++)
            {
                var thing = sut.GetOrAdd(key, () => GetStuff(), GetOptions());
                Assert.That(thing, Is.Not.Null);
                await Task.Delay(2 * refreshInterval);
            }

            // we refreshed every second in 6 seconds so generated 6 times
            // even though we only fetched it every other second which would be 3 times
            Assert.That(timesGenerated, Is.EqualTo(6));
        }

Blasphemous

If I understand the ExpirationMode.ImmediateExpiration behavior correctly, it doesn’t solve the delay for long-running GetStuff method during the gap between the expired time and return of GetStuff. It will be good to have a optional parameter reloadBeforeExpireTimeSpan , that will allow to avoid such gap. I recently suggested similar to Polly (github.com/App-vNext/Polly/issues/794). BTW, the more appropriate name of the mode you added, would be ExpirationMode.ImmediateEviction instead of ExpirationMode.ImmediateExpiration – Adumbrate

Good spot on the typo! Will fix that – Blasphemous

If you need to do premature cache repopulation before the expiry then you need something running in its own thread with its own timer calling the add method on the cache. This should probably be contained in an IHostedService or similar so the host knows about it. Discussed it a bit in github.com/alastairtree/LazyCache/issues/95 – Blasphemous