Is it possible to pre-create a virtualenv for use in hudson builds?

Question

I'm following the outline of the hudson/python/virtualenv CI solution described at heisel.org but one step of this is really chafing, and that's the part where the virtualenv, created just for the CI run, is configured:

pip install -q -E ./ve -r requirements.pip
pip install -q -E ./ve -r requirements-test.pip

This takes an inordinate amount of time to run, and every time a source file changes we'll end up re-downloading what amounts to a significant amount of data.

Is it possible to create template workspaces in Hudson, so that instead of checking out into a bare workspace it checks out into one that is pre-prepared?

Answer 1

Here are a couple options:

1. Have an archive in your source repository that blows up into the virtualenv/pip install. You'll need to make the virtualenv starting point relocatable.

2. Use whatever SCM option is appropriate to not wipe out the workspace between builds (e.g. Use svn update, or don't check Mercurial's Clean Build option). Then keep the install commands in your build script, but put them in under an if statement so they are only run (for example) if a .pip_installed file is not present, or if a build parameter is set.

3. You might be able to get the Clone Workspace plugin to do what you want. But that's an alternative SCM, which I'm guessing you probably don't want since Hudson won't check out from multiple SCMs (see this previous question for some ideas about working around this).

It's probably also a good idea to set up your pip configuration to pull from a local cache of packages.

pip -f http://localhost/packages/