How do I work around this problem creating a virtualenv environment with a custom-build Python?

Question

I need to run some code on a Linux machine with Python 2.3.4 pre-installed. I'm not on the sudoers list for that machine, so I built Python 2.6.4 into (a subdirectory in) my home directory. Then I attempted to use virtualenv (for the first time), but got:

$ Python-2.6.4/python virtualenv/virtualenv.py ENV
New python executable in ENV/bin/python
Could not find platform dependent libraries <exec_prefix>
Consider setting $PYTHONHOME to <prefix>[:<exec_prefix>]
Installing setuptools.........
 Complete output from command /apps/users/dspitzer/ENV/bin/python -c "#!python
\"\"\"Bootstrap setuptoo...

" /apps/users/dspitzer/virtualen...6.egg:
 Could not find platform dependent libraries <exec_prefix>
Consider setting $PYTHONHOME to <prefix>[:<exec_prefix>]
'import site' failed; use -v for traceback
Traceback (most recent call last):
 File "<string>", line 67, in <module>
ImportError: No module named md5
----------------------------------------
...Installing setuptools...done.
Traceback (most recent call last):
 File "virtualenv/virtualenv.py", line 1488, in <module>
  main()
 File "virtualenv/virtualenv.py", line 529, in main
  use_distribute=options.use_distribute)
 File "virtualenv/virtualenv.py", line 619, in create_environment
  install_setuptools(py_executable, unzip=unzip_setuptools)
 File "virtualenv/virtualenv.py", line 361, in install_setuptools
  _install_req(py_executable, unzip)
 File "virtualenv/virtualenv.py", line 337, in _install_req
  cwd=cwd)
 File "virtualenv/virtualenv.py", line 590, in call_subprocess
  % (cmd_desc, proc.returncode))
OSError: Command /apps/users/dspitzer/ENV/bin/python -c "#!python
\"\"\"Bootstrap setuptoo...

" /apps/users/dspitzer/virtualen...6.egg failed with error code 1

Should I be setting PYTHONHOME to some value? (I intentionally named my ENV "ENV" for lack of a better name.)

Not knowing if I can ignore those errors, I tried installing nose (0.11.1) into my ENV:

$ cd nose-0.11.1/
$ ls
AUTHORS    doc/               lgpl.txt     nose.egg-info/  selftest.py*
bin/       examples/          MANIFEST.in  nosetests.1     setup.cfg
build/     functional_tests/  NEWS         PKG-INFO        setup.py
CHANGELOG  install-rpm.sh*    nose/        README.txt      unit_tests/
$ ~/ENV/bin/python setup.py install
Could not find platform dependent libraries <exec_prefix>
Consider setting $PYTHONHOME to <prefix>[:<exec_prefix>]
Traceback (most recent call last):
 File "setup.py", line 1, in <module>
  from nose import __version__ as VERSION
 File "/apps/users/dspitzer/nose-0.11.1/nose/__init__.py", line 1, in <module>
  from nose.core import collector, main, run, run_exit, runmodule
 File "/apps/users/dspitzer/nose-0.11.1/nose/core.py", line 3, in <module>
  from __future__ import generators
ImportError: No module named __future__

Any advice?

Answer 1

Have you actually run "make install" on your custom python build? Usually you'll want to do something like

./configure --prefix=/path/to/installdir  (other options)
make
make install

Note Prefix can be any directory you have full write-permissions to, for example I very often use $HOME/apps on shared-hosting environments.

Then run /path/to/installdir/bin/python, not the one from your build directory. This should create the correct variables, and after that you can install virtualenv. Might be best to install virtualenv using its setup.py:

cd virtualenv_source_dir
/path/to/installdir/bin/python setup.py install

This may require installing setuptools first, using the same method.

Then finally:

# Just to be safe
export PATH="/path/to/installdir/bin:$PATH" 

virtualenv ~/ENV
~/ENV/bin/easy_install (and such)

Answer 2

In addition to Crast's suggestion of making sure you actually installed your custom compiled Python, you should also check that the custom Python can actually find its libraries. This is the hint you're getting with the message about PYTHONHOME. The import errors suggest you need to set in your .bashrc or appropriate shell configuration export PYTHONHOME=/path/to/python_installation.

Additionally, when you are trying to tell virtualenv to use a non-default version of python, you need to use the -p,--python flag, e.g.,

virtualenv --python=/path/to/python_installation/bin/python myenv

See also the related question, "Use different Python version with virtualenv".