How to dynamically provide arguments to a class __init__ function based on attributes?

Question

How do tools like dataclasses and pydantic create the __init__() functions for the classes they create?

I know I can use these tools but I want to learn how to use the python metaprogramming to parse my class and create a dynamic init function based on attributes and their types.

an example:

class MyClass:
  x: int
  y: str | None
  z: float = 0.1

  # I don't want to manually make this
  def __init__(self, x: int, y: str | None, z: float = 0.1):
    self.x = x
    self.y = y
    self.z = z

Is it possible to do something similar using metaclasses and type annotations?

Mikesell

Code for the decorator, and the code for modifying the class. – Dafodil

Code that creates the __init__ method. – Unbelief

I can parse through state-of-the-art code and eventually understand it but I suppose its more a question of how tooling would look at that metacode and determine the type structures and signatures and how I can write that stuff too – Mikesell

Answer 1

There are two main approaches:

1. Decorator
2. Metaclass

---

The idea for dataclass is that you need to somehow get all the annotations and build that method manually from a string(I'll show you). In this scenario you have the class object created.

Available tools are:

• typing module has get_type_hints function.
• you can access the class' attributes using __dict__.
• you can use exec() to dynamically create objects. (dataclass literally does this)

from typing import get_type_hints


def my_dataclass(cls):
    fn_definition = "def __init__(self, "

    # building the first line of __init__
    for annotation, type_ in get_type_hints(cls).items():
        type_name = type_.__name__ if hasattr(type_, "__name__") else str(type_)
        fn_definition += f"{annotation}: {type_name}"
        if annotation in cls.__dict__:
            fn_definition += f" = {cls.__dict__[annotation]}"
        else:
            fn_definition += ", "
    fn_definition += "):\n"

    # building the body of __init__
    for annotation in get_type_hints(cls):
        fn_definition += f"\tself.{annotation} = {annotation}\n"
    print(fn_definition)

    # creating the function object:
    namespace = {}
    exec(fn_definition, {}, namespace)
    __init__ = namespace["__init__"]
    cls.__init__ = __init__
    return cls


@my_dataclass
class MyClass:
    x: int
    y: str | None
    z: float = 0.1


obj = MyClass(10, 20)
print(obj.x)
print(obj.y)
print(obj.z)

output:

def __init__(self, x: int, y: str | None, z: float = 0.1):
        self.x = x
        self.y = y
        self.z = z

10
20
0.1

Of course this is a naive approach and has some problems but it gives you the overview of the idea. dataclass and pydantic(which I'll show after) do a lot of other stuff as well.

Another approach which is used by Pydantic is using a metaclass. The difference is that with decorators, the class is created first and then you extract the information you want but in metaclasses the information is available in the namespace parameter of the __new__ "before creating that class": (I won't go to implement it because the concept is the same as before)

class MyMeta(type):
    def __new__(cls, name, bases, namespace):
        type_hints = namespace["__annotations__"]
        default_values = {k: namespace[k] for k in type_hints if k in namespace}
        print(type_hints)
        print(default_values)


class MyClass(metaclass=MyMeta):
    x: int
    y: str | None
    z: float = 0.1

output:

{'x': 'int', 'y': 'str | None', 'z': 'float'}
{'z': 0.1}

Now you can now check the actual source codes and follow their implementation.

Also note that there are many helpful functions in inspect module that let's you inspect many information from objects in runtime.

Obligatory

This is incredibly helpful, thank you – Mikesell