unittest import error with virtualenv + google-app-engine-django

Question

I'm working with google-app-engine-django + zipped django. Just running "python manage.py test" succeeded without error.

But with virtualenv, test was failed with "import unittest error". same error with Django 1.1.

- OSX 10.5.6
- google-app-engine-django (r101 via svn) : r100 was failed with launcher 1.3.0
- GoogleAppLauncher 1.3.0
- Django 1.1 & 1.1.1 (zipped) : both failed
- virtualenv 1.4.5 
- virtualenvwrapper 1.24

Error Message:

(django_appengine)Reiot:warclouds Reiot$ python manage.py test
WARNING:root:Could not read datastore data from /var/folders/UZ/UZ1vQeLFH2ShHk4kIiLcFk+++TI/-Tmp-/django_google-app-engine-django.datastore
INFO:root:zipimporter('/Volumes/data/Documents/warclouds/django.zip', 'django/core/serializers/')
.WARNING:root:Can't open zipfile /Users/Reiot/.virtualenvs/django_appengine/lib/python2.5/site-packages/setuptools-0.6c11-py2.5.egg: IOError: [Errno 13] file not accessible: '/Users/Reiot/.virtualenvs/django_appengine/lib/python2.5/site-packages/setuptools-0.6c11-py2.5.egg'
WARNING:root:Can't open zipfile /Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/setuptools-0.6c9-py2.5.egg: IOError: [Errno 13] file not accessible: '/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/setuptools-0.6c9-py2.5.egg'
ERROR:root:Exception encountered handling request
Traceback (most recent call last):
  File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/tools/dev_appserver.py", line 3177, in _HandleRequest
    self._Dispatch(dispatcher, self.rfile, outfile, env_dict)
  File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/tools/dev_appserver.py", line 3120, in _Dispatch
    base_env_dict=env_dict)
  File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/tools/dev_appserver.py", line 515, in Dispatch
    base_env_dict=base_env_dict)
  File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/tools/dev_appserver.py", line 2379, in Dispatch
    self._module_dict)
  File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/tools/dev_appserver.py", line 2289, in ExecuteCGI
    reset_modules = exec_script(handler_path, cgi_path, hook)
  File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/tools/dev_appserver.py", line 2185, in ExecuteOrImportScript
    exec module_code in script_module.__dict__
  File "/Volumes/data/Documents/warclouds/main.py", line 28, in <module>
    from appengine_django import InstallAppengineHelperForDjango
  File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/tools/dev_appserver.py", line 1264, in Decorate
    return func(self, *args, **kwargs)
  File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/tools/dev_appserver.py", line 1914, in load_module
    return self.FindAndLoadModule(submodule, fullname, search_path)
  File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/tools/dev_appserver.py", line 1264, in Decorate
    return func(self, *args, **kwargs)
  File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/tools/dev_appserver.py", line 1816, in FindAndLoadModule
    description)
  File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/tools/dev_appserver.py", line 1264, in Decorate
    return func(self, *args, **kwargs)
  File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/tools/dev_appserver.py", line 1767, in LoadModuleRestricted
    description)
  File "/Volumes/data/Documents/warclouds/appengine_django/__init__.py", line 44, in <module>
    import unittest
ImportError: No module named unittest
INFO:root:"GET / HTTP/1.1" 500 -
INFO:root:zipimporter('/Users/Reiot/.virtualenvs/django_appengine/lib/python2.5/site-packages/setuptools-0.6c11-py2.5.egg', '')
INFO:root:zipimporter('/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/setuptools-0.6c9-py2.5.egg', '')
F...........................................................
======================================================================
FAIL: a request to the default page works in the dev_appserver
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Volumes/data/Documents/warclouds/appengine_django/tests/integration_test.py", line 176, in testBasic
    self.assertEquals(rv.status_code, 200)
AssertionError: 500 != 200

I also tried with console import but it was ok.

> which python
/Users/Reiot/.virtualenvs/django_appengine/bin/python
> python
>>> import unittest

Here is my environments:

$ mkvirtualenv --no-site-packages no-django
$ mkvirtualenv --no-site-packages django-1.1
$ mkvirtualenv --no-site-packages django-1.1.1
(django-1.1)$ easy_install Django-1.1.tar
(django-1.1.1)$ easy_install Django-1.1.1.tar
$ mkdir google-app-engine-django-svn
$ cp -r google-app-engine-django-svn google-app-engine-django-svn-django-1.1
// copy appropriate django.zip
$ cp -r google-app-engine-django-svn google-app-engine-django-svn-django-1.1.1
// copy appropriate django.zip

Answer 1

This is quite complex to achieve. I'm researching how to create a virtualenv - GAE - django combination right now, and I might come back with more info later but this is what I have right now:

A difference here is that I'm using Django-nonrel, but I have tried google-app-engine-django too, and in respect to virtualenv they seem to work the same.

The virtualenv'd python interpreter won't be able to start manage.py directly: django's manage.py and dev_appserver.py wants extensive control over sys.path, obviously in a way that makes coexistence with virtualenv possible.

However, a certain level of co-existence seems to be possible:

I have created a directory structure like this:

./src <-- django-nonrel sources and my sources djangoappengine/ <-- from wkornewald's bitbucket sources django/ <-- also wkornewald's bitbucket.. / <-- our application

./www <-- my django root manage.py, app.yaml, settings.py, templates/ etc.. common-apps/ <-- django-nonrel wants packages here .google_appengine <-- link to my installation of GAE

I start the development server with 'python2.5 manage.py runserver' in the directory called 'www/' above.

(basically the directory structure suggested by rh0dium at http://rh0dium.blogspot.com/2010/02/development-strategy-for-google-app.html )

I create my virtualenv in the top directory above:

$ virtualenv --python=python2.5 --no-site-packages --distribute .

Then I put two files in lib/python2.5/site-packages:

gae.pth,

<full path to GAE>
<full path to GAE>/lib/antlr3
<full path to GAE>/lib/webob
<full path to GAE>/lib/yaml/lib

and django-nonrel.pth:

<project path>/src/django-nonrel
<project path>/src

(Those files could/(should?) be separated, but I keep it simple for now.

Doing this in combination with a properly set up setup.py to create develop-eggs at least let me have an interactive prompt that can import everything that the Django setup can import, for playing and testing.