views:

117

answers:

1

I have a package named jiva_tasks, which I'm trying to import via celery (using the CELERY_IMPORTS attribute of celeryconfig. The import statement that celery is using is this:

__import__(module, [], [], [''])

Oddly enough, when this syntax is used, the module gets imported twice, once as jiva_tasks and another time as jiva_tasks. (with a period at the end). Now, chances are good that celery should be passing in globals rather than an empty list, but this seems broken to me. It seems odd that even if given the wrong arguments, __import__ would import something that isn't a valid python module name.

I know that the way to fix this is to pass in globals, but I want to understand why I'm getting this result. Is this a bug, or is there something I don't understand about how __import__ is working?

Update: It also seems to be working fine if I use importlib.

Update 2: Here's the sys.meta_path and sys.import_path right before the __import__ line gets executed:

>>> sys.meta_path
[]
>>> sys.path_hooks
[<type 'zipimport.zipimporter'>]

It doesn't appear to me that there's anything out of the ordinary. However, I just now realized that the package I'm importing is installed using setuptools' develop command. Does that make a difference?

+1  A: 

Creating an empty file "foo.py" and then creating a "bar.py" that says:

__import__('foo', [], [], [''])
import sys
print sorted(sys.modules)

prints out a list that only includes foo once, and not foo. or anything else with a trailing dot — so it's not simply the fact that celery is using __import__ that is causing the problem here. Can you figure out what additional step celery is taking that adds the second module? (Or does a print sorted(sys.modules) right before and after the __import__() call suggest that both modules appear at that moment — in which case we need to look at what import hooks are defined at the moment of the import?)

Brandon Craig Rhodes
I didn't think about import hooks. Although `foo.` *does* seem to be getting registered in sys.modules when that `__import__` call is made.
Jason Baker
If the `__import__()` adds `foo.` then let's look at the import hooks! Add to your question the output of printing `sys.metapath` and `sys.path_hooks` and, if you find any, try sharing their code with us.
Brandon Craig Rhodes
Updated. It doesn't look like I have any import hooks that are out of the ordinary.
Jason Baker
I was going to ask what the value of `module` is at the moment that celery performs the import, on the off chance that it was equal to `'jiva_tasks.'` with a trailing period already, and thinking that such a value might inspire the Python import logic to first import `jiva_tasks` before trying `jiva_tasks.` with a period, per The Rules for doing multi-level imports. But I have just tried an `__import__('foo.', [], [], [''])` in my `bar.py` script above, and it works fine — only `foo` gets imported. `'foo..'` or more periods gives an error, but one is fine, interestingly enough!
Brandon Craig Rhodes