Handling relations between multiple subversion projects

Question

In my company we are using one SVN repository to hold our C++ code. The code base is composed from a common part (infrastructure and applications), and client projects (developed as plugins).

The repository layout looks like this:

• Infrastructure
• App1
• App2
• App3
• project-for-client-1
  • App1-plugin
  • App2-plugin
  • Configuration
• project-for-client-2
  • App1-plugin
  • App2-plugin
  • Configuration

A typical release for a client project includes the project data and every project that is used by it (e.g. Infrastructure).

The actual layout of each directory is -

• Infrastructure
  • branches
  • tags
  • trunk
• project-for-client-2
  • branches
  • tags
  • trunk

And the same goes for the rest of the projects.

We have several problems with the layout above:

1. It's hard to start a fresh development environment for a client project, since one has to checkout all of the involved projects (for example: Infrastructure, App1, App2, project-for-client-1).
2. It's hard to tag a release in a client projects, for the same reason as above.
3. In case a client project needs to change some common code (e.g. Infrastructure), we sometimes use a branch. It's hard to keep track which branches are used in projects.

Is there any way in SVN to solve any of the above? I thought of using svn:externals in the client projects, but after reading this post I understand it might not be right choice.

Answer 1

From my experience I think it's more useful to have a repository for each individual project. Otherwise you have the problems you say and furthermore, revision numbers change if other projects change which could be confusing.

Only when there's a relation between individual projects, such as software, hardware schematics, documentation, etc. We use a single repository so the revision number serves to get the whole pack to a known state.

Answer 2

Yeah, it sucks. We do the same thing, but I can't really think of a better layout.

So, what we have is a set of scripts that can automate everything subversion related. Each customer's project will contain a file called project.list, which contains all of the subversion projects/paths needed to build that customer. For example:

Infrastructure/trunk
LibraryA/trunk
LibraryB/branches/foo
CustomerC/trunk

Each script then looks something like this:

for PROJ in $(cat project.list); do
    # execute commands here
done

Where the commands might be a checkout, update or tag. It is a bit more complicated than that, but it does mean that everything is consistent, checking out, updating and tagging becomes a single command.

And of course, we try to branch as little as possible, which is the most important suggestion I can possibly make. If we need to branch something, we will try to either work off of the trunk or the previously-tagged version of as many of the dependencies as possible.

Answer 3

A suggestion is to change directory layout from

• Infrastructure
  • branches
  • tags
  • trunk
• project-for-client-1
  • branches
  • tags
  • trunk
• project-for-client-2
  • branches
  • tags
  • trunk

to

• branches
  • feature-1
    • Infrastructure
    • project-for-client-1
    • project-for-client-2
• tags
• trunk
  • Infrastructure
  • project-for-client-1
  • project-for-client-2

There are some problems with this layout too. Branches become massive, but at least it's easier to tag specific places in your code.

To work with the code, one would simply checkout the trunk and work with that. Then you don't need the scripts that check out all the different projects. They just refer to Infrastructure with "../Infrastructure". Another problem with this layout is that you need to checkout several copies if you want to work on projects completely independently. Otherwise a change in Infrastructure for one project might cause another project not to compile until that is updated too.

This might make releases a little more cumbersome too, and separating the code for different projects.

Answer 4

You could handle this with svn:externals. This is the url to a spot in an svn repo This lets you pull in parts of a different repository (or the same one). One way to use this is under project-for-client2, you add an svn:externals link to the branch of infrastructure you need, the branch of app1 you need, etc. So when you check out project-for-client2, you get all of the correct pieces.

The svn:externals links are versioned along with everything else, so as project-for-client1 get tagged, branched, and updated the correct external branches will always get pulled in.

Answer 5

First, I don't agree that externals are evil. Although they aren't perfect.

At the moment you're doing multiple checkouts to build up a working copy. If you used externals it would do exactly this, but automatically and consistently each time.

If you point your externals to tags (and or specific revisions) within the target projects, you only need to tag the current project per release (as that tag would indicate exactly what external you were pointing to). You'd also have a record within your project of exactly when you changed your externals references to use a new version of a particular library.

Externals aren't a panacea - and as the post shows there can be problems. I'm sure there's something better than externals, but I haven't found it yet (even conceptually). Certainly, the structure you're using can yield a great deal of information and control in your development process, using externals can add to that. However, the problems he had weren't fundamental corruption issues - a clean get would resolve everything, and are pretty rare (are you really unable to create a new branch of a library in your repo?).

Points to consider - using recursive externals. I'm not sold on either the yes or no of this and tend to go with a pragmatic approach.

Consider using piston as the article suggests, I've not seen it in action so can't really comment, it may do the same job as externals in a better way.