flask-cache memoize URL query string parameters as well

Question

The flask-cache extension has a @cache.memoize decorator to cache a view including the view's *args and **kwargs. Some of my views however take a URL query string as well, for example /foo/image?width=640. The decorator adds a make_cache_key method to the decorated view function that can be used to customise the cache key

However I do not know how to get the request.args outside of the normal request context.

Any ideas how to make the @cache.memoize work with URL query strings as well?

Mccullers

You can extract image generation function from view and cache it's results. – Salep

Yes, that did the trick. – Mccullers

This is a ridiculously frustrating answer, as it comes up in a search for using flask-cache & how to manage routes with url params, but does not provide an actual answer. – Monomial

Answer 1

You can use flask-caching:

Continuation of the Flask-Cache Extension

With which you can do something like this:

@app.route("/")
@cache.cached(timeout=10, query_string=True)
def index():
    return render_template('index.html')

Docs from source code:

:param query_string: Default False. When True, the cache key
                     used will be the result of hashing the
                     ordered query string parameters. This
                     avoids creating different caches for
                     the same query just because the parameters
                     were passed in a different order. See
                     _make_cache_key_query_string() for more
                     details.

Dowery

I just upvote both your comment in GithHub and this answer ;) – Gomer

Answer 2

I had the same problem today and didn't find any example on the internet so I played around a little.

This is my make_cache_key:

def make_cache_key(*args, **kwargs):
    path = request.path
    args = str(hash(frozenset(request.args.items())))
    lang = get_locale()
    return (path + args + lang).encode('utf-8')

You could use request.url instead of path and the hashed args. I needed to add the users language to the key as well.

Caching a view:

@app.route("/test")
@cache.cached(timeout=50)
def test():
    a = request.args.get('a')
    b = request.args.get('b')
    return a + b
test.make_cache_key = make_cache_key

It works but i think it's kind of cumbersome. It turned out, that the key_prefix can be a callable which generates the whole cache_key. Therefore we can do this:

@app.route("/test2")
@cache.cached(timeout=50, key_prefix=make_cache_key)
def test2():
    a = request.args.get('a')
    b = request.args.get('b')
    return a + b

I just came up with this and haven't used it in production yet – so it may not work in all cases.

Weitzel

Thanks! This worked for me: def make_cache_key(*args, **kwargs): return request.url – Adiabatic

This could lead to some buildup of duplicate data in redis could it not? If you request say, /comments?blah=blah, and /comments?foo=foo, both of which yield the same results because for instance the args are ignored, the responses will both get cached. – Slant

@Slant This is a valid point! But may not present a practical issue for typical users (they aren't necessarily mucking with URLs). In any case a similarly dynamic way could be done, with stronger consistency: def make_cache_key(*args, **kwargs): return (request.path,) + (arg for arg in args) + (entry for entry in sorted(kwargs.items())) (untested, just a starting point) – Mussorgsky

In order to support list parameter you may transform the ImmutableMultiDict returned from request.args into a dict of sets: {k, set(s) for k, s in request.args.lists()} But the suggestion by @EyalLevin is much better: https://mcmap.net/q/459383/-flask-cache-memoize-url-query-string-parameters-as-well. – Anarthria

I've been looking for this all day! Great answer. Flask-Caching really has some lacking/vague documentation. – Hindoo

Answer 3

Since version 0.3.4, key_prefix can be a callable :

New in version 0.3.4: Can optionally be a callable which takes no arguments but returns a string that will be used as the cache_key.

Here is the doc : Flask-Cache

Thinnish

Answer 4

thanks to Smoe and Asdine El Hrychy, here is my version; it generates a key for the current URL + query string (which, in my opinion, must be a common need).

Note:

• the query string is sorted in an attempt to stay the same if you request ?a=foo&b=bar or ?b=bar&a=foo (initially i did (k, v) for k, val in flask.request.args.viewitems() for v in sorted(val) but I changed to sort the keys too)
• it supports same key with multiple occurrences, ie: ?a=foo&a=bar (and will return the same key as ?a=bar&a=foo)
• key_prefix argument is only for cached and not for memoize, at least as of Flask-Cache 0.13.1, and since it takes the URL path, it should fit most memoize use-cases

The code:

import flask
import urllib

def cache_key():
    args = flask.request.args
    key = flask.request.path + '?' + urllib.urlencode([
        (k, v) for k in sorted(args) for v in sorted(args.getlist(k))
    ])
    return key

# ...
import time

@app.route('/test')
@cache.cached(timeout=600, key_prefix=cache_key)
def test():
    return time.time()

@app.route('/<value>/test')
@cache.cached(timeout=3600, key_prefix=cache_key)
def value_test(value):
    return flask.jsonify(time=time.time(), value=value)

Picture

Answer 5

since I don't want brother myself to do more work like quote the args, but the following code can working and can satisfy my requirement:

from flask import request

def cache_key():
    return request.url

@main.route("/test/", methods=['GET'])
@cache.cached(timeout=10, key_prefix=cache_key)
def do_somthing():
    return "hello %s" % str(request.args)

Statesmanship