220

I know the issue of circular imports in python has come up many times before and I have read these discussions. The comment that is made repeatedly in these discussions is that a circular import is a sign of a bad design and the code should be reorganised to avoid the circular import.

Could someone tell me how to avoid a circular import in this situation?: I have two classes and I want each class to have a constructor (method) which takes an instance of the other class and returns an instance of the class.

More specifically, one class is mutable and one is immutable. The immutable class is needed for hashing, comparing and so on. The mutable class is needed to do things too. This is similar to sets and frozensets or to lists and tuples.

I could put both class definitions in the same module. Are there any other suggestions?

A toy example would be class A which has an attribute which is a list and class B which has an attribute which is a tuple. Then class A has a method which takes an instance of class B and returns an instance of class A (by converting the tuple to a list) and similarly class B has a method which takes an instance of class A and returns an instance of class B (by converting the list to a tuple).

1

3 Answers 3

405

Consider the following example python package where a.py and b.py depend on each other:

/package
    __init__.py
    a.py
    b.py

Types of circular import problems

Circular import dependencies typically fall into two categories depending on what you're trying to import and where you're using it inside each module. (And whether you're using python 2 or 3).

1. Errors importing modules with circular imports

In some cases, just importing a module with a circular import dependency can result in errors even if you're not referencing anything from the imported module.

There are several standard ways to import a module in python

import package.a           # (1) Absolute import
import package.a as a_mod  # (2) Absolute import bound to different name
from package import a      # (3) Alternate absolute import
import a                   # (4) Implicit relative import (deprecated, python 2 only)
from . import a            # (5) Explicit relative import

Unfortunately, only the 1st and 4th options actually work when you have circular dependencies (the rest all raise ImportError or AttributeError). In general, you shouldn't be using the 4th syntax, since it only works in python2 and runs the risk of clashing with other 3rd party modules. So really, only the first syntax is guaranteed to work.

EDIT: The ImportError and AttributeError issues only occur in python 2. In python 3 the import machinery has been rewritten and all of these import statements (with the exception of 4) will work, even with circular dependencies. While the solutions in this section may help refactoring python 3 code, they are mainly intended for people using python 2.

Absolute Import

Just use the first import syntax above. The downside to this method is that the import names can get super long for large packages.

In a.py

import package.b

In b.py

import package.a

Defer import until later

I've seen this method used in lots of packages, but it still feels hacky to me, and I dislike that I can't look at the top of a module and see all its dependencies, I have to go searching through all the functions as well.

In a.py

def func():
    from package import b

In b.py

def func():
    from package import a

Put all imports in a central module

This also works, but has the same problem as the first method, where all the package and submodule calls get super long. It also has two major flaws -- it forces all the submodules to be imported, even if you're only using one or two, and you still can't look at any of the submodules and quickly see their dependencies at the top, you have to go sifting through functions.

In __init__.py

from . import a
from . import b

In a.py

import package

def func():
    package.b.some_object()

In b.py

import package

def func():
    package.a.some_object()

2. Errors using imported objects with circular dependencies

Now, while you may be able to import a module with a circular import dependency, you won't be able to import any objects defined in the module or actually be able to reference that imported module anywhere in the top level of the module where you're importing it. You can, however, use the imported module inside functions and code blocks that don't get run on import.

For example, this will work:

package/a.py

import package.b

def func_a():
    return "a"

package/b.py

import package.a

def func_b():
    # Notice how package.a is only referenced *inside* a function
    # and not the top level of the module.
    return package.a.func_a() + "b"

But this won't work

package/a.py

import package.b

class A(object):
    pass

package/b.py

import package.a

# package.a is referenced at the top level of the module
class B(package.a.A):
    pass

You'll get an exception

AttributeError: module 'package' has no attribute 'a'

Generally, in most valid cases of circular dependencies, it's possible to refactor or reorganize the code to prevent these errors and move module references inside a code block.

Sign up to request clarification or add additional context in comments.

18 Comments

Brendan, this is an amazingly through answer! I have read a hundred answers about solving circular imports, finally I get it. thx! By the way, you might want to add one more solution to your list: set global for each package to None at top of file, then inject module into global at runtime. This has advantage that all module names are at top of file.
You can somewhat combine the first and second approaches by using absolute imports (so you can see all the dependencies at the top) and naming the components you want at point-of-use. Something like import package;def func(): a = package.a in lieu of from package import a
Would deferring import until inside a function also be slower, in case the function is repeatedly called? Or does the import only occur once?
I'm curious about the statement that this has been fixed in Python 3. Do you have a good reference describing the way that this has changed? I know that there have been changes in that area between 2 and 3, but a small example of the from...import form fails in the same way on both Python 2.7.11 and Python 3.5.1.
How about if 'module_name' not in sys.modules: import ... ?
|
178

Only import the module, don't import from the module:

Consider a.py:

import b

class A:
    def bar(self):
        return b.B()

and b.py:

import a

class B:
    def bar(self):
        return a.A()

This works perfectly fine.

7 Comments

Wow!! Thanks, I knew the trick putting one of the "from imports" causing the circular import error at the bottom of the module, but this is better!
Doesn't seem to work with submodules import foobar.mod_a and import foobar.mod_b doesn't work like described above.
Also this has a big disadvantage: You end up with run-time errors instead of import-time errors when you e.g. delete a function and forget to update a reference to it somewhere.
Community: This this answer does not suffice. please read Brenden's below!
@Nick it does, except that you would have to use foobar.mod_a.function_foo (assuming function_foo exists in mod_a). In any case, I am not recommending, just pointing it out.
|
10

We do a combination of absolute imports and functions for better reading and shorter access strings.

  • Advantage: Shorter access strings compared to pure absolute imports
  • Disadvantage: a bit more overhead due to extra function call

main/sub/a.py

import main.sub.b
b_mod = lambda: main.sub.b

class A():
    def __init__(self):
        print('in class "A":', b_mod().B.__name__)

main/sub/b.py

import main.sub.a
a_mod = lambda: main.sub.a

class B():
    def __init__(self):
        print('in class "B":', a_mod().A.__name__)

5 Comments

Why use the lambda? Why not just b_mod = main.sub.b?
@BrendanAbel The lambda delays access until when it needs to be used. Without it, an AttributeError is raised
Please don't assign lambda expressions to bound names. Use a classic def instead.
Note that wrapping a_mod in a function can have some unexpected effects. For example, type(a_mod) always returns <class 'function'>, regardless of what the type of main.sub.a is. Docstrings also won't work as expected.
Why not just alias with as? You can import like import main.sub.b as b_mod and use the shorter b_mod. Similar for the a_mod

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.