Using Hudson and build steps with multiple git repositories

Question

I'm trying out Hudson to replace our current Buildbot setup. I installed the git plugin. Our current setup is like:

ssh://server:/repo/test_framework.git
ssh://server:/repo/project_a.git

Now, to build project_a I added a new job with multiple git repositories (the ones above). I wanted Hudson to clone the repositories into different directories under $WORKSPACE, becase test_framework needs that hierarchy. But Hudson seems to merge everything into $WORKSPACE instead. From the console log:

warning: no common commits
...
[workspace] $ git merge-base ce14a4579e87971659e5e0469136713847055a29 96d2b3c27595de243702414c4358366923696d78
[workspace] $ git merge-base ce14a4579e87971659e5e0469136713847055a29 5bb011b3fa288afd5e4392640b32b8bcc982103e
[workspace] $ git merge-base ce14a4579e87971659e5e0469136713847055a29 aa6ade81669883909ba5f5459a205df1bd0df3c0

Can I configure this in Hudson to better fit our project setup? Do I need to create a local dummy git repository with every project as git submodules or something?

Answer 1

Within Hudson you can chain multiple jobs together. You could try creating separate Hudson jobs for test_framework and another for project_a. Hudson creates a separate directory in $WORKSPACE for each job, so now you should have two different directories under $WORKSPACE.

---

Setup Chaining

In the job configuration of project_a scroll down to Post-build actions and check Build other projects... Enter in test_framework as the project to build.

In the job configuration of test_framework ensure that Poll SCM is unchecked and that Build after other projects is set to project_a.

---

How it works

What you have now configured is project_a will poll the SCM looking for changes, when changes are found it will pull them from git. Run build steps (if any) and on completion trigger the test_framework job to pull changes from git (if any) and run its build steps.

Answer 2

The problem with the "Build other projects" solution is that if there are changes to test_framework it will not trigger project_a to build. Instead, I recommend abandoning the git plugin and setting up an "Execute shell" build step with the following:

rm -rf ${WORKSPACE}/*

git clone ssh://server:/repo/test_framework.git ${WORKSPACE}/test_framework
cd ${WORKSPACE}/test_framework
git fetch -t ssh://user@server:/repo/test_framework.git +refs/heads/*:refs/remotes/origin/*
git ls-tree HEAD

git clone ssh://server:/repo/project_a.git ${WORKSPACE}/project_a
cd ${WORKSPACE}/project_a
git fetch -t ssh://user@server:/repo/project_a.git +refs/heads/*:refs/remotes/origin/*
git ls-tree HEAD

Next, create hook files "server:/repo/test_framework.git/hooks/post-receive" and "server:/repo/project_a.git/hooks/post-receive" with the following content:

#!/bin/sh
curl http://hudson/job/job_name/build

Now, whenever changes are pushed to either repository, the hook will use Hudson's API to trigger a build.

Answer 3

I ran into the same problem and currently solved it by creating a job for each project and using the Copy Artifact Plugin to allow building the dependent job even if a Git update is done on it's dependencies (this is to avoid building in the middle of an update to the project which we depend upon).

So project_a would copy the latest stable artifacts it needs from test_framework and an update to test framework would trigger a build in project_a. project_a can still be triggered by a change in Git, it just copies again the latest artifacts in test_framework.