What happens when using mutual or circular (cyclic) imports?

Question

In Python, what happens when two modules attempt to import each other? More generally, what happens if multiple modules attempt to import in a cycle?

---

_{See also What can I do about "ImportError: Cannot import name X" or "AttributeError: ... (most likely due to a circular import)"? for the common problem that may result, and advice on how to rewrite code to avoid such imports. See Why do circular imports seemingly work further up in the call stack but then raise an ImportError further down? for technical details on why and how the problem occurs.}

Seamy

See also stackoverflow.com/questions/158268/… – Contrail

also just as a reference, it seems circular imports are allowed on python 3.5 (and probably beyond) but not 3.4 (and probably bellow). – Desert

I'm using python 3.7.2 and am still having a runtime error because of circular dependencies. – Provisional

@CharlieParker This applies specifically to relative imports, according to What's new in 3.5. The relevant issue tracker entry is here. Changes were also made in 3.7 to support some absolute import cases. However, this doesn't prevent AttributeErrors - it enables looking up the partially initialized module in sys.modules, but doesn't resolve time paradoxes. – Brantbrantford

Answer 1

There was a really good discussion on this over at comp.lang.python last year. It answers your question pretty thoroughly.

Imports are pretty straightforward really. Just remember the following:

'import' and 'from xxx import yyy' are executable statements. They execute when the running program reaches that line.

If a module is not in sys.modules, then an import creates the new module entry in sys.modules and then executes the code in the module. It does not return control to the calling module until the execution has completed.

If a module does exist in sys.modules then an import simply returns that module whether or not it has completed executing. That is the reason why cyclic imports may return modules which appear to be partly empty.

Finally, the executing script runs in a module named __main__, importing the script under its own name will create a new module unrelated to __main__.

Take that lot together and you shouldn't get any surprises when importing modules.

Cussedness

@meawoppl Could you expand this comment, please? How specifically have they changed? – Litchfield

As of now, the only reference to circular imports in python3 "What's new?" pages is in the 3.5 one. It says "Circular imports involving relative imports are now supported". @meawoppl have you found anything else what is not listed in these pages? – Gitt

They are def. not supported in 3.0-3.4. Or at least the semantics for success are different. Here is a synopsis I found that dosn't mention the 3.5 changes. gist.github.com/datagrok/40bf84d5870c41a77dc6 – Ratfink

Please can you expand on this "Finally, the executing script runs in a module named main, importing the script under its own name will create a new module unrelated to main.". So lets say the file is a.py and when it runs as main entry point, itsbthe main now if it has code like from a import some variable. Then will the same file 'a.py' get loaded in the sys modules table? So does it mean that if it has say print statement then it will run twice? Once for the main file and again when import is encountered? – Vanessa

This answer is 10 years old, and i would like a modernized update to ensure that it remains correct in various versions of Python, 2.x or 3.x – Noisy

@Fallenreaper: It remains entirely correct as of 3.11. The fix for the issue with circular imports and relative imports doesn't change the correctness of the answer. – Finicky

@variable: When you do python a.py, the sys.modules table will spawn an entry for a module named '__main__' but not for 'a'. If the code in a.py imports b, and b then imports a, with no other imports you end up with three entries in sys.modules, for '__main__', 'b' and 'a'. The code run to define __main__ and a is the same (an import guard of the form if __name__ == '__main__': in a.py could change how the code runs), but they'd be independent modules. If a.py's code defined class Spam, two independent copies would exist, one on __main__, one on a. – Finicky

Answer 2

If you do import foo (inside bar.py) and import bar (inside foo.py), it will work fine. By the time anything actually runs, both modules will be fully loaded and will have references to each other.

The problem is when instead you do from foo import abc (inside bar.py) and from bar import xyz (inside foo.py). Because now each module requires the other module to already be imported (so that the name we are importing exists) before it can be imported.

Examples of working circular imports in Python 2 and Python 3

The article When are Python circular imports fatal? gives four examples when circular imports are, for the reason explained above, nonfatal.

Top of module; no from; Python 2 only

# lib/foo.py                         # lib/bar.py
import bar                           import foo

def abc():                           def xyz():
    print(bar.xyz.__name__)              print(foo.abc.__name__)

Top of module; from ok; relative ok; Python 3 only

# lib/foo.py                         # lib/bar.py
from . import bar                    from . import foo

def abc():                           def xyz():
    print(bar.xyz.__name__)              print(abc.__name__)

Top of module; no from; no relative

# lib/foo.py                         # lib/bar.py
import lib.bar                       import lib.foo

def abc():                           def xyz():
    print(lib.bar.xyz.__name__)          print(lib.foo.abc.__name__)

Bottom of module; import attribute, not module; from okay

# lib/foo.py                         # lib/bar.py
def abc():                           def xyz():
    print(xyz.__name__)                  print(abc.__name__)


from .bar import xyz                 from .foo import abc

Top of function; from okay

# lib/foo.py                         # lib/bar.py
def abc():                           def xyz():
    from . import bar                    from . import foo
    print(bar.xyz.__name__)              print(foo.abc.__name__)

Additional examples

The article cited above does not discuss star imports.

Margalo

It seems that from foo import * and from bar import * will also work fine. – Boughton

Check the edit to the post above using a.py/b.py. He does not use from x import y, and yet still gets the circular import error – Haematosis

This is not entirely true. Just like import * from, if you attempt to access an element in the circular import, at the top level, so before the script completes its run, then you will have the same issue. For instance if you are setting a package global in one package from another, and they both include each other. I was doing this to create a sloppy factory for an object in the base class where that object could be one of a number of subclasses and the using code did not need to be aware of which it was actually creating. – Lederhosen

@Boughton Not really. That will only import the names that are available when the import statement is executed. So it won't error out but you may not get all the variables you expect. – Dupuis

Note, if you do from foo import * and from bar import *, everything executed in the foo is in the initializing phase of bar, and the actual functions in bar has not yet been defined... – Frustum

@Margalo Interesting. I absolutely though that importing specific functions from a module was the solution, as opposed to importing the whole module. One would think that a single independent function gets rid of the problem, no? – Sylvanite

@Martian2049 What does the second part of the answer mean? - "The problem..." - I didn't understand. Please help. – Radiancy

Answer 3

Cyclic imports terminate, but you need to be careful not to use the cyclically-imported modules during module initialization.

Consider the following files:

a.py:

print "a in"
import sys
print "b imported: %s" % ("b" in sys.modules, )
import b
print "a out"

b.py:

print "b in"
import a
print "b out"
x = 3

If you execute a.py, you'll get the following:

$ python a.py
a in
b imported: False
b in
a in
b imported: True
a out
b out
a out

On the second import of b.py (in the second a in), the Python interpreter does not import b again, because it already exists in the module dict.

If you try to access b.x from a during module initialization, you will get an AttributeError.

Append the following line to a.py:

print b.x

Then, the output is:

$ python a.py
a in                    
b imported: False
b in
a in
b imported: True
a out
Traceback (most recent call last):
  File "a.py", line 4, in <module>
    import b
  File "/home/shlomme/tmp/x/b.py", line 2, in <module>
    import a
 File "/home/shlomme/tmp/x/a.py", line 7, in <module>
    print b.x
AttributeError: 'module' object has no attribute 'x'

This is because modules are executed on import and at the time b.x is accessed, the line x = 3 has not be executed yet, which will only happen after b out.

Tourane

this greatly explains the problem, but how about the solution? how could we properly import and print x? the other solution above did not work for me – Gradey

I think this answer would benefit a lot if you used __name__ instead of 'a'. At the beginning, I was totally confused why a file would be executed twice. – Vapid

@Gradey Refactor your project so the import statements form a tree-like structure (main script imports supporting modules which themselves may import their supporting modules etc.). This is the generally advisable approach. – Sideling

Answer 4

As other answers describe this pattern is acceptable in python:

def dostuff(self):
     from foo import bar
     ...

Which will avoid the execution of the import statement when the file is imported by other modules. Only if there is a logical circular dependency, this will fail.

Most Circular Imports are not actually logical circular imports but rather raise ImportError errors, because of the way import() evaluates top level statements of the entire file when called.

These ImportErrors can almost always be avoided if you positively want your imports on top:

Consider this circular import:

App A

# profiles/serializers.py

from images.serializers import SimplifiedImageSerializer

class SimplifiedProfileSerializer(serializers.Serializer):
    name = serializers.CharField()

class ProfileSerializer(SimplifiedProfileSerializer):
    recent_images = SimplifiedImageSerializer(many=True)

App B

# images/serializers.py

from profiles.serializers import SimplifiedProfileSerializer

class SimplifiedImageSerializer(serializers.Serializer):
    title = serializers.CharField()

class ImageSerializer(SimplifiedImageSerializer):
    profile = SimplifiedProfileSerializer()

From David Beazleys excellent talk Modules and Packages: Live and Let Die! - PyCon 2015, 1:54:00, here is a way to deal with circular imports in python:

try:
    from images.serializers import SimplifiedImageSerializer
except ImportError:
    import sys
    SimplifiedImageSerializer = sys.modules[__package__ + '.SimplifiedImageSerializer']

This tries to import SimplifiedImageSerializer and if ImportError is raised, because it already is imported, it will pull it from the importcache.

PS: You have to read this entire post in David Beazley's voice.

Threadgill

ImportError is not raised if module have been already imported. Modules can be imported as many times as you want to i.e. "import a; import a;" is ok. – Hobbie

this would make it a module instead of class from my experiment. – Camarata

Answer 5

To my surprise, no one has mentioned cyclic imports caused by type hints yet.
If you have cyclic imports only as a result of type hinting, they can be avoided in a clean manner.

Consider main.py which makes use of exceptions from another file:

from src.exceptions import SpecificException

class Foo:
    def __init__(self, attrib: int):
        self.attrib = attrib

raise SpecificException(Foo(5))

And the dedicated exception class exceptions.py:

from src.main import Foo

class SpecificException(Exception):
    def __init__(self, cause: Foo):
        self.cause = cause

    def __str__(self):
        return f'Expected 3 but got {self.cause.attrib}.'

This will raise an ImportError since main.py imports exception.py and vice versa through Foo and SpecificException.

Because Foo is only required in exceptions.py during type checking, we can safely make its import conditional using the TYPE_CHECKING constant from the typing module. The constant is only True during type checking, which allows us to conditionally import Foo and thereby avoid the circular import error.
Something to note is that by doing so, Foo is not declared in exceptions.py at runtime, which leads to a NameError. To avoid that, we add from __future__ import annotations which transforms all type annotations in the module to strings.

Hence, we get the following code for Python 3.7+:

from __future__ import annotations
from typing import TYPE_CHECKING
if TYPE_CHECKING:  # Only imports the below statements during type checking
   ​from src.main import Foo

class SpecificException(Exception):
   def __init__(self, cause: Foo):  # Foo becomes 'Foo' because of the future import
       ​self.cause = cause

   ​def __str__(self):
       ​return f'Expected 3 but got {self.cause.attrib}.'

In Python 3.6, the future import does not exist, so Foo has to be a string:

from typing import TYPE_CHECKING
if TYPE_CHECKING:  # Only imports the below statements during type checking
   ​from src.main import Foo

class SpecificException(Exception):
   ​def __init__(self, cause: 'Foo'):  # Foo has to be a string
       ​self.cause = cause

   ​def __str__(self):
       ​return f'Expected 3 but got {self.cause.attrib}.'

In Python 3.5 and below, the type hinting functionality did not exist yet.
In future versions of Python, the annotations feature may become mandatory, after which the future import will no longer be necessary. Which version this will occur in is yet to be determined.

This answer is based on Yet another solution to dig you out of a circular import hole in Python by Stefaan Lippens.

Madrigal

FYI, from __future__ import annotations was planned to become the default in 3.10, deferred to 3.11, and then put on indefinite hold due to some problems it creates that they haven't come up with a good solution for. So if you want it, you're going to need the __future__ import for awhile. – Finicky

are there any downsides to wrapping 'Foo' in string, and forgo importing annotations? – Magnetism

That depends on your IDE. If you are relying on type annotations in an IDE/linter, it may not be able to resolve, leaving you in the same position you were before the annotations. – Phlogopite

This fixed my issue. As I recently added quite a few type hints I'm not sure which one caused the error as Python doesn't give enough feedback in the stack trace. – Peabody

Answer 6

Module a.py :

import b
print("This is from module a")

Module b.py

import a
print("This is from module b")

Running "Module a" will output:

>>> 
'This is from module a'
'This is from module b'
'This is from module a'
>>> 

It output this 3 lines while it was supposed to output infinitival because of circular importing. What happens line by line while running"Module a" is listed here:

1. The first line is import b. so it will visit module b
2. The first line at module b is import a. so it will visit module a
3. The first line at module a is import b but note that this line won't be executed again anymore, because every file in python execute an import line just for once, it does not matter where or when it is executed. so it will pass to the next line and print "This is from module a".
4. After finish visiting whole module a from module b, we are still at module b. so the next line will print "This is from module b"
5. Module b lines are executed completely. so we will go back to module a where we started module b.
6. import b line have been executed already and won't be executed again. the next line will print "This is from module a" and program will be finished.

Gleeman

this might happen just because a.py, when executed as a script, will be named "module __main__", not "module a". So when it gets to b and encounters import a, now it will import the same file under a different module name, right? What happens when neither are the __main__ script? – Panicstricken

@MestreLion: Your understanding is correct; running python a.py or python -ma will import a.py's code twice, first as __main__, then (while importing b into __main__) as a, and the top-level code executes twice to create two independent modules in the cache (if you defined a class or function in a.py, there would be two instances with the same code, but unrelated identities for type checking purposes). If neither are the __main__ script, then (in Mohsen's design) you'd see one print from each module (the print from the module you directly imported would appear second). – Finicky

Answer 7

I got an example here that struck me!

foo.py

import bar

class gX(object):
    g = 10

bar.py

from foo import gX

o = gX()

main.py

import foo
import bar

print "all done"

At the command line: $ python main.py

Traceback (most recent call last):
  File "m.py", line 1, in <module>
    import foo
  File "/home/xolve/foo.py", line 1, in <module>
    import bar
  File "/home/xolve/bar.py", line 1, in <module>
    from foo import gX
ImportError: cannot import name gX

Seamy

How did you fix this? I'm trying to understand circular import to fix a problem of my own that looks very similar to what you're doing... – Deloris

Errm... I think I fixed my problem with this incredibly ugly hack. {{{ if not 'foo.bar' in sys.modules: from foo import bar else: bar = sys.modules['foo.bar'] }}} Personally, I think circular imports are a HUGE warning sign on bad code design... – Deloris

@c089, or you could just move import bar in foo.py to the end – Ailsun

If bar and foo both must use gX, the 'cleanest' solution is to put gX in another module and have both foo and bar import that module. (cleanest in the sense that there are no hidden semantic dependencies.) – Beadroll

Tim has a good point. Basically it's because bar can't even find gX in the foo. the circular import is fine by itself, but it's just that gX is not defined when it's imported. – Frustum

Answer 8

There are a lot of great answers here. While there are usually quick solutions to the problem, some of which feel more pythonic than others, if you have the luxury of doing some refactoring, another approach is to analyze the organization of your code, and try to remove the circular dependency. You may find, for example, that you have:

File a.py

from b import B

class A:
    @staticmethod
    def save_result(result):
        print('save the result')

    @staticmethod
    def do_something_a_ish(param):
        A.save_result(A.use_param_like_a_would(param))
    
    @staticmethod
    def do_something_related_to_b(param):
        B.do_something_b_ish(param)

File b.py

from a import A

class B:
    @staticmethod
    def do_something_b_ish(param):
        A.save_result(B.use_param_like_b_would(param))

In this case, just moving one static method to a separate file, say c.py:

File c.py

def save_result(result):
    print('save the result')

will allow removing the save_result method from A, and thus allow removing the import of A from a in b:

Refactored File a.py

from b import B
from c import save_result

class A:
    @staticmethod
    def do_something_a_ish(param):
        save_result(A.use_param_like_a_would(param))
    
    @staticmethod
    def do_something_related_to_b(param):
        B.do_something_b_ish(param)

Refactored File b.py

from c import save_result

class B:
    @staticmethod
    def do_something_b_ish(param):
        save_result(B.use_param_like_b_would(param))

In summary, if you have a tool (e.g. pylint or PyCharm) that reports on methods that can be static, just throwing a staticmethod decorator on them might not be the best way to silence the warning. Even though the method seems related to the class, it might be better to separate it out, especially if you have several closely related modules that might need the same functionality and you intend to practice DRY principles.

Ness

Answer 9

I completely agree with pythoneer's answer here. But I have stumbled on some code that was flawed with circular imports and caused issues when trying to add unit tests. So to quickly patch it without changing everything you can resolve the issue by doing a dynamic import.

# Hack to import something without circular import issue
def load_module(name):
    """Load module using imp.find_module"""
    names = name.split(".")
    path = None
    for name in names:
        f, path, info = imp.find_module(name, path)
        path = [path]
    return imp.load_module(name, f, path[0], info)
constants = load_module("app.constants")

Again, this isn't a permanent fix but may help someone that wants to fix an import error without changing too much of the code.

Cheers!

Astatine

Answer 10

Circular imports can be confusing because import does two things:

1. it executes imported module code
2. adds imported module to importing module global symbol table

The former is done only once, while the latter at each import statement. Circular import creates situation when importing module uses imported one with partially executed code. In consequence it will not see objects created after import statement. Below code sample demonstrates it.

Circular imports are not the ultimate evil to be avoided at all cost. In some frameworks like Flask they are quite natural and tweaking your code to eliminate them does not make the code better.

main.py

print 'import b'
import b
print 'a in globals() {}'.format('a' in globals())
print 'import a'
import a
print 'a in globals() {}'.format('a' in globals())
if __name__ == '__main__':
    print 'imports done'
    print 'b has y {}, a is b.a {}'.format(hasattr(b, 'y'), a is b.a)

b.by

print "b in, __name__ = {}".format(__name__)
x = 3
print 'b imports a'
import a
y = 5
print "b out"

a.py

print 'a in, __name__ = {}'.format(__name__)
print 'a imports b'
import b
print 'b has x {}'.format(hasattr(b, 'x'))
print 'b has y {}'.format(hasattr(b, 'y'))
print "a out"

python main.py output with comments

import b
b in, __name__ = b    # b code execution started
b imports a
a in, __name__ = a    # a code execution started
a imports b           # b code execution is already in progress
b has x True
b has y False         # b defines y after a import,
a out
b out
a in globals() False  # import only adds a to main global symbol table 
import a
a in globals() True
imports done
b has y True, a is b.a True # all b objects are available

Thimbleweed

Answer 11

Suppose you are running a test python file named request.py In request.py, you write

import request

so this also most likely a circular import.

Solution:

Just change your test file to another name such as aaa.py, other than request.py.

Do not use names that are already used by other libs.

Aney

This is my best answer as my issue was just that I've named the file similar to the name of the lib I'm importing from. – Equate

Answer 12

I solved the problem the following way, and it works well without any error. Consider two files a.py and b.py.

I added this to a.py and it worked.

if __name__ == "__main__":
        main ()

a.py:

import b
y = 2
def main():
    print ("a out")
    print (b.x)

if __name__ == "__main__":
    main ()

b.py:

import a
print ("b out")
x = 3 + a.y

The output I get is

>>> b out 
>>> a out 
>>> 5

Toreador

Answer 13

Ok, I think I have a pretty cool solution. Let's say you have file a and file b. You have a def or a class in file b that you want to use in module a, but you have something else, either a def, class, or variable from file a that you need in your definition or class in file b. What you can do is, at the bottom of file a, after calling the function or class in file a that is needed in file b, but before calling the function or class from file b that you need for file a, say import b Then, and here is the key part, in all of the definitions or classes in file b that need the def or class from file a (let's call it CLASS), you say from a import CLASS

This works because you can import file b without Python executing any of the import statements in file b, and thus you elude any circular imports.

For example:

File a:

class A(object):

     def __init__(self, name):

         self.name = name

CLASS = A("me")

import b

go = B(6)

go.dostuff

File b:

class B(object):

     def __init__(self, number):

         self.number = number

     def dostuff(self):

         from a import CLASS

         print "Hello " + CLASS.name + ", " + str(number) + " is an interesting number."

Voila.

Arium

from a import CLASS doesn't actually skip executing all the code in a.py. This is what really happens: (1) All the code in a.py gets run as a special module "__main__". (2) At import b, the top-level code in b.py gets run (defining class B) and then control returns to "__main__". (3) "__main__" eventually passes control to go.dostuff(). (4) when dostuff() comes to import a, it runs all the code in a.py again, this time as the module "a"; then it imports the CLASS object from the new module "a". So actually, this would work equally well if you used import a anywhere in b.py. – Doggish

Answer 14

This issue arose in my case when one of my python file was the same name as a library. Example you have a package 'abc' and file 'abc.py'. import abc will throw a cyclic error.

Paltry

Answer 15

I encounter this error too. In my case, I fixed it by swapping import-statements in the __init__.py

My situiation

init.py

from .b import B, C
form .a import A

b.py

from <module name of __init__.py> import A

a.py

class A():
  a = 42

My solution

init.py

form .a import A
from .b import B, C

Autotruck

Answer 16

bar.py

print('going to import foo')
from foo import printx

foo.py

print('trying to import bar')
import bar

def printx():
    print('x')

$ python bar.py

going to import foo
trying to import bar
going to import foo
Traceback (most recent call last):
  File "bar.py", line 2, in <module>
    from foo import printx
  File "/home/suhail/Desktop/temp/circularimport/foo.py", line 2, in <module>
    import bar
  File "/home/suhail/Desktop/temp/circularimport/bar.py", line 2, in <module>
    from foo import printx
ImportError: cannot import name 'printx' from partially initialized module 'foo' (most likely due to a circular import) (/home/suhail/Desktop/temp/circularimport/foo.py)

Anemophilous

Not sure how this is attempting to answer the question. Random examples of problems with circular imports are not answering the general question of how circular imports work. – Finicky

@Finicky i just tried to recreate when does the circular error happens – Anemophilous