Python 3 type hinting for dynamic attributes through __getattr__ [duplicate]

Question

To wrap all function calls of a module and access it via an wrapper class's __getattr__ method, I tried to use typing library but I could not figure out on how to do this correctly.

import interface

"""
>>> print(interface.__all__)
['execute_foo_operation', ...]
"""

class InterfaceWrapper(object):
    def __init__(self, job_queue):
        self.job_queue = job_queue
        self.callbacks = []

    def __getattr__(self, name):
        func = getattr(interface, name)
        return functools.partial(self._wrapper, func)

    def _wrapper(self, func, *args, **kwargs):
        job = func(*args, **kwargs)
        self.job_queue.push(job)
        for callback in self.callbacks:
            callback(job)
        return job

    def register_callback(self, callback):
        self.callbacks.append(callback)


class Operator(object):
    def __init__(self, job_queue):
        self.interface = InterfaceWrapper(job_queue)

    def after_queuing(self):
        # do something

    def execute_foo_operation(self):
        self.interface.register_callback(self.after_queuing)
        self.interface.execute_foo_operation()  # unresolved attribute

Can anyone guide my code to work correctly?

Hypertensive

Could you elaborate on which methods or expressions you would like to type-annotate? If it's InterfaceWrapper.__getattr__, I think it would be hard to do so since it requires knowing the argument types of func, which I guess are different for each function. – Almetaalmighty

Answer 1

Usually, __getattr__ can return anything. So you could use Typing.Any

From Typing import Any

def __getattr__(self, name: str) -> Any:
    ...

But it looks like your implementation of __getattr__ is to return a callable

From Typing import Callable

def __getattr__(self, name: str) -> Callable:
    ...

Diary

Note that avoiding Any is usually better. If you instead annotate the return value as object, then code that uses this will be forced to type check (or, at worst, cast) the result, and the following code will be type-safe(r). If you use Any then nothing using this return value will be safe, even though you'll get no error for it. – Bigener

Answer 2

If I understand correctly, what you request is essentially to ask the statical type checker to infer the reference of dynamical attributes returned by __getattr__, and this is probably quite challenging for most of the typing inspectors. In fact, for PyCharm (>=2020.3.5 (203.7717.81)) the inspection of unresolved attributes is disabled if __getattr__ is defined in the class.

A workaround exists, yet requires you to not use __getattr__ to access the attributes but statically defined all attribute names manually via typing.Literal before hand and use these Literal to annotate the name in the custom defined get_attr method below:

import functools
from typing import Literal
supported = Literal[Literal['a'], Literal['b']]


class InterfaceWrapper(object):
    def __init__(self, job_queue):
        self.job_queue = job_queue
        self.callbacks = []

    def get_attr(self, name: supported):
        func = getattr(interface, name)
        return functools.partial(self._wrapper, func)

    def _wrapper(self, func, *args, **kwargs):
        job = func(*args, **kwargs)
        self.job_queue.push(job)
        for callback in self.callbacks:
            callback(job)
        return job

    def register_callback(self, callback):
        self.callbacks.append(callback)

class Operator(object):
    interface: InterfaceWrapper

    def __init__(self, job_queue):
        self.interface: InterfaceWrapper = InterfaceWrapper(job_queue)

    def after_queuing(self):
        # do something
        ...

    def execute_foo_operation(self):
        # ... do something
        self.interface.get_attr('a')()

Herein if you call let's say you call self.interface.get_attr('c') it will report that 'a' and 'b' are expected but get 'c' instead.

If you only want to access the methods or attributes through __getattr__ (interface.attr_name) then you would need at least statically define the attributes of interface somewhere in the InterfaceWrapper, e.g., define the attributes of the same name that points to the corresponding fields in the interface object.

Holocaust