Typical SVN repo structure seems to be sub-optimal for continuous integration...

Question

I've set up our SVN repository like the Subversion book suggests, and this is also how my previous companies have done it. It looks something like this:

/trunk
/branches
/tags
/extlibs
/docs

where the first three are pretty obvious, and extlibs is for 3rd party assemblies that we wouldn't typically recompile ourselves.

All of this works great for the daily development stuff.

Now I've installed TeamCity and have builds, unit tests, code coverage, and code analysis running. Everything is great, except for the fact that this code structure results in too much code getting downloaded.

So here's the catch 22, in my opinion: it's silly to download all of aforementioned folders from the SVN repo when I only need /trunk and /extlibs. But I can only specify one repo folder to download in the TeamCity VCS settings. So then the other possibility is to put the /extlibs folder into /trunk, but in order to compile branches, /extlibs would have to go into all of those as well (since I usually branch the trunk, and not individual subfolders... and this would seem infinitely more evil since /extlibs could actually be larger than /trunk and /branches, with all of the binaries stored there...

Do you guys have any suggestions for me? Thanks!

Answer 1

Set TeamCity to download only /trunk. Make /trunk/extlibs an external that points to /extlibs. Do the same for all /branches. This would allow you to share /extlibs between /trunk and all /branches, while still having single root folder for each of them.

Update: When I said to make /extlibs an external in /trunk/extlibs, I was referring to the SVN externals property. Assuming your SVN server is svn://yoursvnserver/svn, you can do this with the following svn command:

svn.exe propset svn:externals svn://yoursvnserver/svn/trunk -F externals_defs.txt

You can refer to svn.exe help propset for more details on the format of the file, but in the simplest form, it'll have one line like this:

extlibs svn://yoursvnserver/svn/extlibs

Once you set the SVN externals property on the trunk, you'll commit it. From there on, everybody that checks out /trunk (including TeamCity) will get in /trunk/extlibs a copy of /extlibs. Of course, you will have to change your references a bit so that they point to /trunk/extlibs now, so that your build is self-contained in the /trunk tree.

Answer 2

One approach is to store the extlibs in their own "project" structure. Treat them like a seperate project that your main project references instead of a part of the main project. Then set TeamCity to "build" these binaries(which is simply to pull them down) when they are modified. Since your main project is dependent on these binaries, when they are "built" it should trigger a build of the projects that depend on them. That way you aren't pulling down the binaries each time you get ready to build your main project but only as they are recompiled and committed into SVN.

Answer 3

I'd say that putting extlibs in the trunk is actually a good idea, that not only solves your problem but another as well.

What happens when you need to release a previous tagged build but you've updated some of the libraries in extlibs since then? You have to fish through the history to find a version of extlibs that was current when that build was tagged.

Answer 4

It looks like you have two different issues you need to address:

1. How to set up your VCS for your build configuration
2. How to handle your external libraries

For your VCS, leave your setup as you have it - at the root of your repository. In each build configuration, you can add checkout rules to prune your checkout to the right folders.

For instance, you can use the following to make the trunk folder the root folder of the checkout:

+:trunk=>.

Or for a branch, you can do something like:

+:branches/featureset=>.

You can add multiple checkout rules if desired, including adding any files/folders from your extlib folder:

+:extlibs/dependency-2.5.3=>.libs/dependency1

You can even remove files/folders using checkout rules:

-:trunk/files-not-needed 

For your external libs, as stated in other answers, you can include external folders using SVN. However, I would not get heavy handed with the inclusion of the entire extlibs folder. Only include the specific library revisions that you need to build the project. Also, make sure your extlibs folders are well-protected from change. By being fine-grained about your externals, you can avoid the issue of trying to build an old version because your project will be linked to the correct version. If you have triggered builds enabled, you will also avoid a library change triggering a build. For example, you could configure your snv:external libs like so:

libs/dependency1 svn://yoursvnserver/svn/extlibs/dependency-2.5.3
libs/dependency2 svn://yoursvnserver/svn/extlibs/dependency-rc-2.4

Answer 5

I'd like to chip in a smallish answer as well. All the answers here about how to instruct Subversion to keep a "current" copy of the external libraries as part of trunk is basically the right way to go forward in this particular case.

However, if for some reason you want to pull down parts of a repository in TeamCity, like you asked, you can do that as well.

When you edit the build configuration, on the second tab, the one for version control settings, you have a setting for each attached VCS root, for checkout rules.

Click on the "edit checkout rules" link to the right in the VCS root grid for that build configuration and enter the following (as per your example):

-:.
+:trunk
+:extlibs

This will first flag everything as "do not pull down", both files and subfolders of the root, and then flag both trunk and extlibs as "do pull these down".

You can read more about the checkout rules here.