Migrating from CVS(NT) to Subversion: What's the equivalent of (virtual) modules?

Question

We are currently considering to move away from CVSNT, very probably to Subversion (because we are already using trac, which is well-prepared for SVN-integration). As we have been making rather extensive use of some less common CVS(NT)-features a couple of questions arose very early on. This is but the first of them.

Here's the scenario: (For the impatient: My actual question is at the very bottom of this post below the horizontal ruler.)

All of our "projects" share some common code. The directory hierarchy inside the CVS repository looks somewhat like this (simplified - hopefully not too much so):

• Libs
  • Lib1
  • Lib2
• External
  • ExtLib1
    • Docs
    • Source
  • ExtLib2
    • Docs
    • Source
• ProjectA
• ProjectB

...where ProjectA might for example be dependent on Lib1, Lib2, ExtLib1 and ExtLib2 and ProjectB dependent on Lib1 and ExtLib1 (actually this might be because Lib1 itself depends on ExtLib1). We modelled this by using the CVSROOT/modules file like so:

ExtLib1Full -d Lib/ExtLib1Full    External/ExtLib1
ExtLib1Src  -d Lib/ExtLib1/Source External/ExtLib1/Source
ExtLib2Full -d Lib/ExtLib2Full    External/ExtLib2
ExtLib2Src  -d Lib/ExtLib2/Source External/ExtLib2/Source

Lib1        -d Lib/Lib1           Libs/Lib1
Lib1_WDeps  -a ExtLib1Src Lib1
Lib2        -d Lib/Lib2           Libs/Lib2
Lib2_WDeps  -a Lib2

ProjectAMain -a ProjectA
ProjectALibs -a Lib1_WDeps Lib2_WDeps ExtLib2
ProjectAFull -a ProjectAMain ProjectALibs

ProjectALibs_OursOnly -a Lib1 Lib2
ProjectAFull_OursOnly -a ProjectAMain ProjectALibs_OursOnly

ProjectBMain -a ProjectB
ProjectBLibs -a Lib1_WDeps
ProjectBFull -a ProjectBMain ProjectBLibs

ProjectBLibs_OursOnly -a Lib1
ProjectBFull_OursOnly -a ProjectBMain ProjectBLibs_OursOnly

For a build of ProjectA the build server would now simply have to check out the virtual module named "ProjectAFull" to get all interdependent modules needed for that particular build - and even create the directory structure favoured by the compiler (i.e. external and internal libs both placed below a common "Lib" parent folder). Likewise when we want to tag a release including all dependencies we would simply use cvs rtag -rTagName ProjectAFull. When producing a ChangeLog we would use the output of cvs rlog ProjectAFull_OursOnly in order not to let the ChangeLog be polluted by commit messages from the external libraries (which we maintain using cvs import and vendor branches).

---

Is there an equivalent of these virtual modules in SVN? How should I set up the directory structure inside the new SVN repository to accommodate for this? Should each project and library become its own SVN project (i.e. with its own set of "trunk", "tags" and "branches" folders) or should I simply import the existing directory structure as-is? How do I define the dependencies?
I would very much prefer to continue to be able to do single-step tag/checkout/log operations that would only affect the relevant modules.

Answer 1

Don't forget to look at Mercurial or Git as well if you're making the change. The future is DVCS after all.

EDIT: There is a lot of ignorance surrounding DVCS as evidenced by the down votes on this comment but I stand by my suggestion that it be looked at because the question states that a source control migration is being considered and that Subversion is a candidate. I'm afraid I'll have to go down with the ship on this one.

I realise that many svn users are still smarting from Linus Torvalds saying that it was the most pointless project ever (on camera), but insults aside, the merits of DVCS have been strong enough to see adoption of Mercurial by the likes of Mozilla, Google Code and others. Even the Subversion project authors acknowledge that there are a growing number of projects that cannot benefit from subversion over dvcs.

Answer 2

Have you looked at SVN externals yet? They seem to be what you're looking for.

Answer 3

You should look at SVN Externals, but note that I am not sure that I understand what you're doing with CVS, since I haven't used CVS in years.

Basically, an external reference means you can create a sub-directory beneath your project, and check out another project into that.

Basically, you can get this layout:

ProjectA      - svn://server/ProjectA
  classes      (svn://server/ProjectA/classes)
  app          (svn://server/ProjectA/app)
  externals    (svn://server/ProjectA/externals)
    Ext1      - svn://server/Ext1
      classes  (svn://server/Ext1/classes)
    Ext2      - svn://server/Ext2

Explanation: - svn://... is the url I checked out, (svn://...) is the relative url of that directory, but it came as part of the checkout of the mother project directory.

Note that the external references are added as properties to the "externals" directory, and committed into the repository as part of ProjectA. After you've added those properties, an update, or a fresh checkout, will automatically download both Ext1 and Ext2 as part of the normal checkout.

This will give you working folders inside working folders, however, and you'll need to commit changes and branch/tag those separately. Subversion does not allow you to do modifications to all those working copies and commit them all in one step.

In other words, if in order to add a feature to the project, you change some files in ProjectA/classes, and then add some framework support on Ext1/classes, you'll have to execute two commits, one for Ext1 and one for ProjectA.

---

If you want a live example, here's a sub-directory of my C# class library: http://vkarlsen.serveftp.com:81/LVK/LVK_3_5/trunk/LVK.UnitTests

To checkout that directory, you'll need to specify a username and password, both should be 'guest' without the quotes.

External references in my class library unit test project includes both the "SigningKey" directory (check subversion properties on the main directory), as well as the contents of the "libs" sub-directory (check subversion properties on libs). As you noted, when you checked out the unit test project, it pulled down everything else it needed of libraries, well, except for the projects it tests.

However, if I now add a unit test, and simultaneously update the version of the SQLite library files, I need to do 2 commits. One to get the new SQLite files into the repository, because it is a separate project/working folder, and one to get the new unit test into the repository.