detecting circular imports

Question

I'm working with a project that contains about 30 unique modules. It wasn't designed too well, so it's common that I create circular imports when adding some new functionality to the project.

Of course, when I add the circular import, I'm unaware of it. Sometimes it's pretty obvious I've made a circular import when I get an error like AttributeError: 'module' object has no attribute 'attribute' where I clearly defined 'attribute'. But other times, the code doesn't throw exceptions because of the way it's used.

So, to my question:

Is it possible to programmatically detect when and where a circular import is occuring?

The only solution I can think of so far is to have a module importTracking that contains a dict importingModules, a function importInProgress(file), which increments importingModules[file], and throws an error if it's greater than 1, and a function importComplete(file) which decrements importingModules[file]. All other modules would look like:

import importTracking
importTracking.importInProgress(__file__)
#module code goes here.
importTracking.importComplete(__file__)

But that looks really nasty, there's got to be a better way to do it, right?

Answer 1

Not all circular imports are a problem, as you've found when an exception is not thrown.

When they are a problem, you'll get an exception the next time you try to run any of your tests. You can change the code when this happens.

I don't see any change required from this situation.

Example of when it's not a problem:

a.py

import b
a = 42
def f():
  return b.b

b.py

import a
b = 42
def f():
  return a.a

Answer 2

import uses __builtin__.__import__(), so if you monkeypatch that then every import everywhere will pick up the changes. Note that a circular import is not necessarily a problem though.

Answer 3

To avoid having to alter every module, you could stick your import-tracking functionality in a import hook, or in a customized __import__ you could stick in the built-ins -- the latter, for once, might work better, because __import__ gets called even if the module getting imported is already in sys.modules, which is the case during circular imports.

For the implementation I'd simply use a set of the modules "in the process of being imported", something like (pseudocode but not too far from real code...):

beingimported = set()
originalimport = __import__
def newimport(modulename, etc etc):
  if modulename in beingimported: BADABOOM!!!
  beingimported.add(modulename)
  result = originalimport(modulename, etc etc)
  beingimported.remove(modulename)
  return result
import __builtin__
__builtin__.__import__ = newimport

Answer 4

Circular imports in python are not like PHP includes.

Python imported modules are loaded first time into an import "handler", and kept there for the duration of the process. This handler assigns names in the local namespace for whatever is imported from that module, for every subsequent import. A module is unique, and a reference to that module name will always point to the same loaded module, regardless of where it was imported.

So if you have a circular module import, the loading of each file will happen once, and then each module will have names relating to the other module created into its namespace.

There could of course be problems when referring to specific names within both modules (when the circular imports occur BEFORE the class/function definitions that are referenced in the imports of the opposite modules), but you'll get an error if that happens.