Python class member lazy initialization

Question

I would like to know what is the python way of initializing a class member but only when accessing it, if accessed. I tried the code below and it is working but is there something simpler than that?

class MyClass(object):

    _MY_DATA = None

    @staticmethod
    def _retrieve_my_data():
        my_data = ...  # costly database call
        return my_data

    @classmethod
    def get_my_data(cls):
        if cls._MY_DATA is None:
            cls._MY_DATA = MyClass._retrieve_my_data()
        return cls._MY_DATA

Counterfeit

Answer 1

You could use a @property on the metaclass instead:

class MyMetaClass(type):
    @property
    def my_data(cls):
        if getattr(cls, '_MY_DATA', None) is None:
            my_data = ...  # costly database call
            cls._MY_DATA = my_data
        return cls._MY_DATA


class MyClass(metaclass=MyMetaClass):
    # ...

This makes my_data an attribute on the class, so the expensive database call is postponed until you try to access MyClass.my_data. The result of the database call is cached by storing it in MyClass._MY_DATA, the call is only made once for the class.

For Python 2, use class MyClass(object): and add a __metaclass__ = MyMetaClass attribute in the class definition body to attach the metaclass.

Demo:

>>> class MyMetaClass(type):
...     @property
...     def my_data(cls):
...         if getattr(cls, '_MY_DATA', None) is None:
...             print("costly database call executing")
...             my_data = 'bar'
...             cls._MY_DATA = my_data
...         return cls._MY_DATA
... 
>>> class MyClass(metaclass=MyMetaClass):
...     pass
... 
>>> MyClass.my_data
costly database call executing
'bar'
>>> MyClass.my_data
'bar'

This works because a data descriptor like property is looked up on the parent type of an object; for classes that's type, and type can be extended by using metaclasses.

Responsible

note that this works at the instance-level, though Etienne wanted something at the class-level. It seems you can't combine properties and classmethods easily. – Cachalot

Ok, but I don't want to cache the result for a particular instance. Instead, I want to cache the result for the class itself because the value is the same for all instances. – Counterfeit

I did not see your previous comments, sorry. – Counterfeit

@EtienneRouxel: Does this solve your problem? Is there anything else you want me to add to the explanation? – Responsible

@MartijnPieters: I am actually looking at what a MetaClass is in order to understand your answer ;-) Thank you by the way. – Counterfeit

@EtienneRouxel: Read What is a metaclass in Python? to get more info about metaclasses than you ever wanted. :-) – Responsible

@MartijnPieters: So if I understand well, the purpose of the meta class is to add the class method "my_data" to all classes based on / build with (whatever the term) this meta class, right? – Counterfeit

@EtienneRouxel: Exactly, and to make my_data work as an attribute on the class. A property is a data descriptor and thus will only work as a class attribute if it is defined on the metaclass. – Responsible

@MartijnPieters: Ok, that made me learned new things, which is good and I thank you for that. However in my real life scenario, my_data is only needed in MyClass so I don’t necessary need the property syntactic sugar. I might stick to my code example by using a private _get_my_data class method which I consider simpler than using meta classes. Thank you very you for your time. – Counterfeit

@EtienneRouxel: Not a problem. Using a class method for this is perfectly fine too. Mind accepting my answer if you feel it was helpful to you at all? – Responsible

@A-B-B: no, it is not, not when you want to allow for the attribute to be set to None on the class and still have the property see that as 'not set'. Also, in Python 2, hasattr() swallows all exceptions (it's basically a blank try: gettattr(obj, attrname) except: setup) which is not always desirable. – Responsible

@MartijnPieters is there any preference of doing this over setting the class attribute directly using property ? like if self.__class__._my_data is None: self.__class__._my_data=something; return self.__class__._mydata – Radman

@dnit13: that would only work on instances. You can't use it on the class as property objects don't bind to the class they are defined on (they return the function object rather that call it). – Responsible

Answer 2

This answer is for a typical instance attribute/method only, not for a class attribute/classmethod, or staticmethod.

For Python 3.8+, how about using the cached_property decorator? It memoizes.

from functools import cached_property

class MyClass:

    @cached_property
    def my_lazy_attr(self):
        print("Initializing and caching attribute, once per class instance.")
        return 7**7**8

For Python 3.2+, how about using both property and lru_cache decorators? The latter memoizes.

from functools import lru_cache

class MyClass:

    @property
    @lru_cache()
    def my_lazy_attr(self):
        print("Initializing and caching attribute, once per class instance.")
        return 7**7**8

---

Credit: answer by Maxime R.

Herrmann

Answer 3

Another approach to make the code cleaner is to write a wrapper function that does the desired logic:

def memoize(f):
    def wrapped(*args, **kwargs):
        if hasattr(wrapped, '_cached_val'):
            return wrapped._cached_val
        result = f(*args, **kwargs)
        wrapped._cached_val = result
        return result
    return wrapped

You can use it as follows:

@memoize
def expensive_function():
    print "Computing expensive function..."
    import time
    time.sleep(1)
    return 400

print expensive_function()
print expensive_function()
print expensive_function()

Which outputs:

Computing expensive function...
400
400
400

Now your classmethod would look as follows, for example:

class MyClass(object):
        @classmethod
        @memoize
        def retrieve_data(cls):
            print "Computing data"
            import time
            time.sleep(1) #costly DB call
            my_data = 40
            return my_data

print MyClass.retrieve_data()
print MyClass.retrieve_data()
print MyClass.retrieve_data()

Output:

Computing data
40
40
40

Note that this will cache just one value for any set of arguments to the function, so if you want to compute different values depending on input values, you'll have to make memoize a bit more complicated.

Cachalot

Answer 4

Consider the pip-installable Dickens package which is available for Python 3.5+. It has a descriptors package which provides the relevant cachedproperty and cachedclassproperty decorators, the usage of which is shown in the example below. It seems to work as expected.

from descriptors import cachedproperty, classproperty, cachedclassproperty

class MyClass:
    FOO = 'A'

    def __init__(self):
        self.bar = 'B'

    @cachedproperty
    def my_cached_instance_attr(self):
        print('Initializing and caching attribute, once per class instance.')
        return self.bar * 2

    @cachedclassproperty
    def my_cached_class_attr(cls):
        print('Initializing and caching attribute, once per class.')
        return cls.FOO * 3

    @classproperty
    def my_class_property(cls):
        print('Calculating attribute without caching.')
        return cls.FOO + 'C'

Herrmann

Answer 5

Ring gives lru_cache-like interface but working with any kind of descriptor supports: https://ring-cache.readthedocs.io/en/latest/quickstart.html#method-classmethod-staticmethod

class Page(object):
    (...)

    @ring.lru()
    @classmethod
    def class_content(cls):
        return cls.base_content

    @ring.lru()
    @staticmethod
    def example_dot_com():
        return requests.get('http://example.com').content

See the link for more details.

Fons