How to get the contents of the cache properly in Django?

Question

I set 4 cache values with LocMemCache as shown below:

from django.core.cache import cache

cache.set("first_name", "John")
cache.set("last_name", "Smith", version=2)
cache.set("age", 36, version=3)
cache.set("gender", "Male")

Then, I tried to see the contents of the cache with cache._cache as shown below:

from django.core.cache import cache

print(cache._cache) # Here

But, the key's values contain b'\x80\x05\x95\x08\... as shown below:

OrderedDict([(':1:gender', b'\x80\x05\x95\x08\x00\x00\x00\x00\x00\x00\x00\x8c\x04Male\x94.'), (':3:age', b'\x80\x05K$.'), (':2:last_name', b'\x80\x05\x95\t\x00\x00\x00\x00\x00\x00\x00\x8c\x05Smith\x94.'), (':1:first_name', b'\x80\x05\x95\x08\x00\x00\x00\x00\x00\x00\x00\x8c\x04John\x94.')])

Actually, the result which I expected is as shown below:

OrderedDict([(':1:gender', 'Male'), (':3:age', 36), (':2:last_name', 'Smith'), (':1:first_name', 'John')])

And, I tried to see the contents of the cache with cache._cache.items() as shown below:

from django.core.cache import cache

print(cache._cache.items()) # Here

But again, the key's values contain b'\x80\x05\x95\x08\... as shown below:

odict_items([(':1:gender', b'\x80\x05\x95\x08\x00\x00\x00\x00\x00\x00\x00\x8c\x04Male\x94.'), (':3:age', b'\x80\x05K$.'), (':2:last_name', b'\x80\x05\x95\t\x00\x00\x00\x00\x00\x00\x00\x8c\x05Smith\x94.'), (':1:first_name', b'\x80\x05\x95\x08\x00\x00\x00\x00\x00\x00\x00\x8c\x04John\x94.')])

Actually, the result which I expected is as shown below:

odict_items([(':1:gender', 'Male'), (':3:age', 36), (':2:last_name', 'Smith'), (':1:first_name', 'John')])

Lastly, I tried to see the contents of the cache with locmem._caches as shown below:

from django.core.cache.backends import locmem

print(locmem._caches) # Here

But again, the key's values contain b'\x80\x05\x95\x08\... as shown below:

{'': OrderedDict([(':1:gender', b'\x80\x05\x95\x08\x00\x00\x00\x00\x00\x00\x00\x8c\x04Male\x94.'), (':3:age', b'\x80\x05K$.'), (':2:last_name', b'\x80\x05\x95\t\x00\x00\x00\x00\x00\x00\x00\x8c\x05Smith\x94.'), (':1:first_name', b'\x80\x05\x95\x08\x00\x00\x00\x00\x00\x00\x00\x8c\x04John\x94.')])}

Actually, the result which I expected is as shown below:

{'': OrderedDict([(':1:gender', 'Male'), (':3:age', 36), (':2:last_name', 'Smith'), (':1:first_name', 'John')])}

My questions:

1. How can I get the contents of the cache properly in Django?
2. What is b'\x80\x05\x95\x08\...?

Santonin

From the django source it's seems like the b'\x80\x05\x95\x08\... denotes the pickled representation of an object – Plosive

This is the pickling. – Rupe

Answer 1

In order to serialize objects, Django will encode data. Indeed, for strings that might not be necessary, but you can also dump a dictionary in it, a class object, or something else.

pickle ^[Python-doc] is a library that can encode (most) objects. It will for objects store the path of the class together with attributes, etc. except if these would for example store file handlers (that can of course not be pickled since that is an open file handler), database connections (not to be confused with connection strings), and all other volatile-like objects, it thus can store nearly everything from simple strings and integers up to very complex objects with (recursive) pointers.

In order to do that, it thus uses a certain protocol. That protocol starts with \x80 identifier, followed by the pickle version (most recent is version 5), and then followed by op-codes and data that will serialize the data, and end with a dot (.) that will pop the object from the stack as the result of serialization.

This thus means that if you store a simple string that will indeed be done with (minimal) overhead. On the other hand you thus can store all sorts of things that are not strings, and pickle can decode these to Python objects: the cache system will do this in a transparent manner.

One could also have been opted for a JSON encoding, or some other encoding, but JSON only would allow encoding certain types (strings, numeric types, booleans, list, dictionaries, and None), and would also fail to encode recursive objects (i.e. a list that contains itself for example).

How can I remove b'\x80\x05\x95\x08\... from the contents of the cache?

You could override the cache to prevent pickling, but that would mean you can now only pickle strings, whereas you might want to pickle for example a database object.

Rupe