How to avoid circular dependency caused by type hinting of pointer attributes in python

Question

Consider the two modules (in the same folder):

firstly, person.py

from typing import List

from .pet import Pet


class Person:
    def __init__(self, name: str):
        self.name = name
        self.pets: List[Pet] = []
    
    def adopt_a_pet(self, pet_name: str):
        self.pets.append(Pet(pet_name, self))

and then pet.py

from .person import Person

    
class Pet:
    def __init__(self, name: str, owner: Person):
        self.name = name
        self.owner = owner

the code above will not work, because of circular dependency. You'll get an error:

ImportError: cannot import name 'Person'

Some ways to make it work:

1. keep the definition of the classes Person and Pet in the same file.
2. do away with the pet.owner attribute (which is there as a convenient pointer)
3. don't use type-hinting / annotation where it would cause circular references:

e.g. just have:

class Pet:
    def __init__(self, name: str, owner):

I see some drawback in all the options I've listed so far.

Is there another way? One that allows me to

• split classes into different files
• use type annotation in combined with pointers such as shown

Or: is there very good reason to instead follow one of the solutions I've already listed?

Eartha

Often it helps instead of from .person import Person to import the module from . import person and use the long name person.Person (same for pet.Pet). The explanation was given here at SO already, don't want to duplicate it. – Quesenberry

Can you point me toward this explanation? I tried your suggestion but I get an error from the pet.py file stating: AttributeError: module 'demo.person' has no attribute 'Person' To me this makes sense because the Pet class is imported during the import of the Person class, so, at the time when Pet is being imported, there is not yet an imported Person class. – Eartha

I remeber following one answer by M.Pieters. The question was mine and the answer explains the difference between dependence on module contents and module existence. Link #36137593 Hope it helps you as it did help me. – Quesenberry

I tried it and got no error when person.py is imported first. – Quesenberry

@VPfb: Could you share the way you did this without getting an error? I test with the following: from demo import person charlie = person.Person('Charlie') charlie.adopt_pet('Lassie') – Eartha

I just imported the module with python3 -m pkg.person. I made the chances from my first comment. – Quesenberry

"chances" -> "changes" – Quesenberry

Answer 1

I ran into similar problems recently and solved it by using the following method:

import typing

if typing.TYPE_CHECKING:
    from .person import Person


class Pet:
    def __init__(self, name: str, owner: 'Person'):
        self.name = name
        self.owner = owner

There is a second solution described here, which requires Python >= 3.7.

from __future__ import annotations  # <-- Additional import.
import typing

if typing.TYPE_CHECKING:
    from .person import Person


class Pet:
    def __init__(self, name: str, owner: Person):  # <-- No more quotes.
        self.name = name
        self.owner = owner

The __future__ import was set to no longer be required as of 3.10, but that has been delayed.

Banneret

Here is the corresponding pep: python.org/dev/peps/pep-0484/#runtime-or-type-checking – Lamb

The apostrophe around Person seems to be optional. It would be nice if someone can explain the difference with or without the apostrophe. – Banneret

The ' are optional in Python > 3.7 or in Python 3.7 if from __future__ import annotations is imported. See python.org/dev/peps/pep-0484/#forward-references and python.org/dev/peps/pep-0563 – Lamb

@Lamb The way I read it from What's New In Python 3.7, the ' are optional in Python >=3.10, or in Python >=3.7 && <3.10 if from __future__ import annotations is used. – Clevelandclevenger

Answer 2

After some more learning, I realized there is a right way to do this: Inheritance:

First I define Person, without [pets] or the method in the OP. Then I define Pets, with an owner of class Person. Then I define

from typing import List
from .person import Person
from .pet import Pet


class PetOwner(Person):
    def __init__(self, name: str):
        super().__init__(name)
        self.pets = []  # type: List[Pet]


    def adopt_a_pet(self, pet_name: str):
        self.pets.append(Pet(pet_name))

All methods in Person that needs to refer to Pet should now be defined in PetOwner and all methods/attributes of Person that are used in Pet need to be defined in Person. If the need arises to use methods/attributes in Pet that are only present in PetOwner, a new child class of Pet, e.g. OwnedPet should be defined.

Of course, if the naming bothers me, I could change from Person and PetOwner to respectively BasePerson and Person or something like that.

Eartha

This is a possible workaround for your use case but does not solve the issue raised by the question. Inheritance isn't always the best model for your data but circular dependencies introduced by type checking still need to be solved. (I upvoted the question though). – Taimi

Answer 3

I had a similar use case of circular dependency error because of type annotation. Consider, the following structure of the project:

my_module  
|- __init__.py (empty file)
|- exceptions.py
|- helper.py

Contents:

# exceptions.py
from .helper import log

class BaseException(Exception):
    def __init__(self):
        log(self)

class CustomException(BaseException):
    pass

# helper.py
import logging
from .exceptions import BaseException

def log(exception_obj: BaseException):
    logging.error('Exception of type {} occurred'.format(type(exception_obj)))

I solved it by using the technique similar to the one described here

Now, the updated content of helper.py looks like the following:

# helper.py
import logging

def log(exception_obj: 'BaseException'):
    logging.error('Exception of type {} occurred'.format(type(exception_obj)))

Note the added quotes in type annotation of exception_obj parameter. This helped me to safely remove the import statement which was causing the circular dependency.

Caution: If you're using IDE (like PyCharm), you still might get suggestion of importing the class and the type hinting by the IDE would not work as expected. But the code runs without any issue. This would be helpful when you want to keep the code annotated for other developers to understand.

Ackerman