I'm having a module import issue.
using python 2.6 on ubuntu 10.10
I have a class that subclasses the daemon at http://www.jejik.com/articles/2007/02/a_simple_unix_linux_daemon_in_python/ . I created a python package with a module containing code that imports some models from a django project. The code works when used from a class, not subclassing the daemon. The structure looks something like:
my_module
__init__.py
- bin
- cfg
- core
__init__.py
collection.py
daemon.py
The ItemRepository code:
class ItemRepository(object):
"""This class provides an implementation to retrieve configuration data
from the monocle database
"""
def __init__(self, project_path):
if project_path is not None:
sys.path.append(project_path)
try:
from django.core.management import setup_environ
from someproj import settings
setup_environ(settings)
from someproj.someapp.models import ItemConfiguration
except ImportError:
print "Could not import models from web app. Please ensure the\
PYTHONPATH is configured properly"
def get_scheduled_collectors(self):
"""This method finds and returns all ItemConfiguration objects
that are scheduled to run
"""
logging.info('At the error path: %s' % sys.path)
# query ItemConfigs from database
items = ItemConfiguration.objects.filter(disabled=False) #ERROR OCCURS AT THIS LINE
return [item for item in items if item.scheduled]
The daemon code (in /usr/local/bin/testdaemon.py):
import sys
from my_module.core.daemon import Daemon
from my_module.core.collection import ItemRepository
import logging
import time
class TestDaemon(Daemon):
default_conf = '/etc/echodaemon.conf'
section = 'echo'
def run(self):
while True:
logging.info('The echo daemon says hello')
ir = ItemRepository(project_path=self.project_path)
items = ir.get_scheduled_collectors() #TRIGGERS ERROR
logging.info('there are %d scheduled items' % len(items))
time.sleep(1)
if __name__ == '__main__':
TestDaemon().main()
The error I get is "NameError: global name 'my_module' is not defined" It get's past the import but then fails when trying to call a method on the object. I'm assuming it has to do with sys.path / PYTHONPATH, but I know my django project is on the path, as I've printed it out. Nothing so far in the python docs or Learning Python has helped yet. Does anyone have any insights or know of a good reference to module imports?
UPDATE:
Now I've attempted to simplify the problem to make it easier to understand. Now I have a directory structure that looks like:
/home
/username
/django
/someproj
/web
models.py
/my_module
daemon.py
I have set the $PYTHONPATH variable in /etc/bash.bashrc to '/home/username/django'.
Inside the testdaemon.py file the imports look like:
import logging
from django.core.management import setup_environ
from someproj import settings
setup_environ(settings)
from someproj.web.models import ItemConfiguration
But now I get an ImportError: No module named 'someproj'. So then I appended the path.
import sys
sys.path.append('/home/username/django')
import logging
from django.core.management import setup_environ
from someproj import settings
setup_environ(settings)
from someproj.web.models import ItemConfiguration
And now the ImportError says: No module named 'web'. Here's the traceback:
Traceback (most recent call last):
File "testdaemon.py", line 77, in <module>
TestDaemon('/tmp/testdaemon.pid').run()
File "testdaemon.py", line 47, in run
scheduled_items = [item for item in ItemConfiguration.objects.filter(disabled=False) if collector.scheduled]
File "/usr/local/lib/python2.6/dist-packages/django/db/models/manager.py", line 141, in filter
return self.get_query_set().filter(*args, **kwargs)
File "/usr/local/lib/python2.6/dist-packages/django/db/models/query.py", line 550, in filter
return self._filter_or_exclude(False, *args, **kwargs)
File "/usr/local/lib/python2.6/dist-packages/django/db/models/query.py", line 568, in _filter_or_exclude
clone.query.add_q(Q(*args, **kwargs))
File "/usr/local/lib/python2.6/dist-packages/django/db/models/sql/query.py", line 1128, in add_q
can_reuse=used_aliases)
File "/usr/local/lib/python2.6/dist-packages/django/db/models/sql/query.py", line 1026, in add_filter
negate=negate, process_extras=process_extras)
File "/usr/local/lib/python2.6/dist-packages/django/db/models/sql/query.py", line 1179, in setup_joins
field, model, direct, m2m = opts.get_field_by_name(name)
File "/usr/local/lib/python2.6/dist-packages/django/db/models/options.py", line 291, in get_field_by_name
cache = self.init_name_map()
File "/usr/local/lib/python2.6/dist-packages/django/db/models/options.py", line 321, in init_name_map
for f, model in self.get_all_related_m2m_objects_with_model():
File "/usr/local/lib/python2.6/dist-packages/django/db/models/options.py", line 396, in get_all_related_m2m_objects_with_model
cache = self._fill_related_many_to_many_cache()
File "/usr/local/lib/python2.6/dist-packages/django/db/models/options.py", line 410, in _fill_related_many_to_many_cache
for klass in get_models():
File "/usr/local/lib/python2.6/dist-packages/django/db/models/loading.py", line 167, in get_models
self._populate()
File "/usr/local/lib/python2.6/dist-packages/django/db/models/loading.py", line 61, in _populate
self.load_app(app_name, True)
File "/usr/local/lib/python2.6/dist-packages/django/db/models/loading.py", line 76, in load_app
app_module = import_module(app_name)
File "/usr/local/lib/python2.6/dist-packages/django/utils/importlib.py", line 35, in import_module
__import__(name)
ImportError: No module named web
So going from the earlier comments I tried adding:
import sys
sys.path.append('/home/username/django')
import logging
from django.core.management import setup_environ
from someproj import settings
setup_environ(settings)
from someproj import web
from someproj.web import models
from someproj.web.models import ItemConfiguration
But that didn't help. So I created a very simple file:
#!/usr/bin/python
import logging
import time
import sys
sys.path.append('/home/username/django')
from django.core.management import setup_environ
from someproj import settings
setup_environ(settings)
from someproj.web.models import ItemConfiguration
if __name__ == '__main__':
print sys.path
items = ItemConfiguration.objects.all()
for item in items:
print item
And this works! Which really only further confuses me. So now I'm thinking maybe it has to do with the daemon. It uses os.fork() and I'm not sure if the path is still set. This is why I set the $PYTHONPATH variable in the /etc/bash.bashrc file.
Any more insights? I really need the daemon, I don't have much of a choice as I need a long running process.