Circular dependency in Python

Question

I have two files, node.py and path.py, which define two classes, Node and Path, respectively.

Up to today, the definition for Path referenced the Node object, and therefore I had done

from node.py import *

in the path.py file.

However, as of today I created a new method for Node that references the Path object.

I had problems when trying to import path.py: I tried it, and when the program ran and called the Path method that uses Node, an exception rose about Node not being defined.

What do I do?

Baileybailie

duplicate? https://mcmap.net/q/20029/-what-happens-when-using-mutual-or-circular-cyclic-imports – Glynda

Are you trying to have one class per file? This is why that rarely works out well. – Jone

Agree with S.Lott. Python is not Java. You don't need one class per file. – Khrushchev

effbot.org/zone/import-confusion.htm – Rus

A couple of people have said "you don't need one class per file" and words to the effect "don't try to be Java". OK - but it's off the point. Class definitions can get very large and bundling them into the same file can make for a very large, unreadable file. In a program I am working on with 8 mutually dependent classes, each of which is several hundred lines in length, I see no benefit in keeping them in the same file and a considerable benefit in keeping them separate. – Isidroisinglass

also just as a reference, it seems circular imports are allowed on python 3.5 (and probably beyond) but not 3.4 (and probably bellow). – Nail

Could not upvote @Isidroisinglass enough.. As if 1,000+ LOC in a multi-class file isn't bad enough to read or maintain, the test file (people do write tests for Python classes, right?) is going to be much longer and even more of a maintenance nightmare. Just because Python sometimes makes good organization difficult doesn't mean we should abandon maintainable code. – Acclivity

Answer 1

One other approach is importing one of the two modules only in the function where you need it in the other. Sure, this works best if you only need it in one or a small number of functions:

# in node.py 
from path import Path
class Node 
    ...

# in path.py
class Path
  def method_needs_node(): 
    from node import Node
    n = Node()
    ...

Carbamate

It would work, but I don't see this passing readability code-review – Compiler

Also keep in mind, that when typing is used, the import needs to be at top level (either start or end of the file). – Secondary

Answer 2

You may not need to import Path in node.py in order for Path and Node to make use of one another.

# in __init__.py  (The order of imports should not matter.)
from .node import Node
from .path import Path

# in path.py 
from . import Node
class Path
  ...

  def return_something_pathy(self): 
    ...

# in node.py
class Node
  def __init__(self, path): 
    self.path = path
    ...

  def a_node_method():
    print(self.path.return_something_pathy())

To make it clear that Node is making use of Path, add type hinting. There is a feature available starting with Python 3.7 to support forward references in type annotations, described in PEP 563.

# in node.py  (Now with type hinting.)
from __future__ import annotations

class Node
  def __init__(self, path: Path): 
    self.path = path
    ...

  def a_node_method():
    print(self.path.return_something_pathy())

I came across a Yet another solution to dig you out of a circular import hole in Python is a great blog post which taught me this.

Euchologion

By itself, a forward reference in type annotation does NOT work in case of absent import in node.py. For make annotations work you still need from .path import Path, but you could place it under if typing.TYPE_CHECKING: branch. With such branch both real execution and linter would work. – Bitstock

Answer 3

I prefer to break a circular dependency by declaring one of the dependencies in the constructor of the other dependent class. In my view this keeps the code neater, and gives easy access to all methods who require the dependency.

So in my case I have a CustomerService and a UserService who depend on each other. I break the circular dependency as follows:

class UserService:

    def __init__(self):
        # Declared in constructor to avoid circular dependency
        from server.portal.services.admin.customer_service import CustomerService
        self.customer_service = CustomerService()

    def create_user(self, customer_id: int) -> User:
        # Now easy to access the dependency from any method
        customer = self.customer_service.get_by_id(customer_id)

Senecal

Here, the problem stays when it originates from one class.inheritance, and can be solved with multiple class inheritance. – Ordinate

Answer 4

Another method is to define them both in the same module, and to delay defining the types. A little like this:

class Node:
   _path_type: type = None
   
   def method_needs_path(self):
       p = self._path_type()
       ...


class Path:
    def method_needs_node(self):
       n = Node()

Node._path_type = Path

It may be nicer to be symmetrical about this:

class Node:
   _path_type: type = None
   
   def method_needs_path(self):
       p = self._path_type()
       ...


class Path:
    _node_type: type = None

    def method_needs_node(self):
       n = Node()

Node._path_type = Path
Path._node_type = Node

This could also be done in multiple modules:

# in node.py
class Node:
   _path_type: type = None
   
   def method_needs_path(self):
       p = self._path_type()
       ...

# in path.py
from .node import Node

class Path:
    _node_type: type = None

    def method_needs_node(self):
       n = self._node_type()

Node._path_type = Path
Path._node_type = Node

# in __init__.py (note that order is important now)
from .node import Node
from .path import Path

Roommate