Virtualenv: Where do I put stuff?

Question

What sort of directory structure should one follow when using virtualenv? For instance, if I were building a WSGI application and created a virtualenv called foobar I would start with a directory structure like:

/foobar
  /bin
    {activate, activate.py, easy_install, python}
  /include
    {python2.6/...}
  /lib
    {python2.6/...}

Once this environment is created, where would one place their own:

• python files?
• static files (images/etc)?
• "custom" packages, such as those available online but not found in the cheese-shop?

in relation to the virtualenv directories?

Answer 1

Heh, we had the same question (pretty much) earlier today!

Answer 2

If you only have a few projects every so often, nothing stops you from creating a new virtualenv for each one, and putting your packages right inside:

/foobar
  /bin
    {activate, activate.py, easy_install, python}
  /include
    {python2.6/...}
  /lib
    {python2.6/...}
  /mypackage1
    __init__.py
  /mypackage2
    __init__.py

The advantage of this approach is that you can always be sure to find find the activate script that belongs to the project inside.

$ cd /foobar
$ source bin/activate
$ python 
>>> import mypackage1
>>>

If you decide to be a bit more organized, you should consider putting all your virtualenvs into one folder, and name each of them after the project you are working on.

  /virtualenvs
    /foobar
      /bin
        {activate, activate.py, easy_install, python}
      /include
        {python2.6/...}
      /lib
        {python2.6/...}
  /foobar
    /mypackage1
      __init__.py
    /mypackage2
      __init__.py

This way you can always start over with a new virtualenv when things go wrong, and your project files stay safe.

Another advantage is that several of your projects can use the same virtualenv, so you don't have to do the same installation over and over if you have a lot of dependencies.

$ cd /foobar
$ source ../virtualenvs/foobar/bin/activate
$ python 
>>> import mypackage2
>>>

For users that regularly have to set up and tear down virtualenvs it would make sense to look at virtualenvwrapper.

http://pypi.python.org/pypi/virtualenvwrapper

With virtualenvwrapper you can

* create and delete virtual environments

* organize virtual environments in a central place

* easily switch between environments

You no more have to worry about where your virtualenvs are when working on the projects "foo" and "bar":

  /foo
    /mypackage1
      __init__.py
  /bar
    /mypackage2
      __init__.py

This is how you start working on project "foo":

$ cd foo
$ workon
bar
foo
$ workon foo
(foo)$ python
>>> import mypackage1
>>>

Then switching to project "bar" is as simple as this:

$ cd ../bar
$ workon bar
(bar)$ python
>>> import mypackage2
>>>

Pretty neat, isn't it?

Answer 3

If you give your project a setup.py, pip can import it from version control directly.

Do something like this:

$ virtualenv --no-site-packages myproject
$ . myproject/bin/activate
$ easy_install pip
$ pip install -e hg+http://bitbucket.org/owner/myproject#egg=proj

The -e will put the project in myproject/src, but link it to myproject/lib/pythonX.X/site-packages/, so any changes you make will get picked up immediately in modules that import it from your local site-packages. The #egg bit tells pip what name you want to give to the egg package it creates for you.

If you don't use --no-site-packages, be careful to specify that you want pip to install into the virtualenv with the -E option

Answer 4

virtualenv provides a python interpreter instance, not an application instance. You wouldn't normally create your application files within the directories containing a system's default python, likewise there's no requirement to locate your application within a virtualenv directory. For example, you might have a project where you have multiple applications using the same virtualenv. Or, you may be testing an application with a virtualenv that will later be deployed with a system python. Or, you may be packaging up a standalone app where it might make sense to have the virtualenv directory located somewhere within the app directory itself. So, in general, I don't think there is one right answer to the question. And, a good thing about virtualenv is that it supports many different use cases: there doesn't need to be one right way.

Answer 5

Because virtualenv's are not relocatable, in my opinion it is bad practice to place your project files inside a virtualenv directory. The virtualenv itself is a generated development/deployment artifact (sort of like a .pyc file), not part of the project; it should be easy to blow it away and recreate it anytime, or create a new one on a new deploy host, etc.

Many people in fact use virtualenvwrapper, which removes the actual virtualenvs from you awareness almost completely, placing them all side-by-side in $HOME/.virtualenvs by default.