How to manage external dependencies which are constantly being modified

Question

Our development uses lots of open-source code and I'm trying to figure out what the best way to manage these external dependencies.

Our current configuration:

• we are developing for both linux and windows
• We use svn for our own code
• external dependencies (boost, log4cpp, etc) are not stored in svn. Instead I put them under ./extern (or c:\extern on windows). I don't want to put them in our repository because I will not be able to update them that way. Some of these are constantly being updated.

My questions

• What to do if I need to modify external code? Currently I have created a folder in my svn repository called extern_hacks and that is where I put the modified external code. I then link (or copy on windows) the files into the external directory structure. This solution is problematic since it is hard to keep track of copying the files, and very hard to update from svn when files are sitting in two repositories (mine for the modified files, and the original repository say sourceforge)

• How to manage versions of external dependencies?

I'm interested to hear how others deal with these issues. Thanks.

Answer 1

I do not understand why you say

I don't want to put them in our repository because I will not be able to update them that way. Some of these are constantly being updated.

You really need to

1. include external dependencies in your source control and periodically update them and then tese, test, test.

2. Coordinate your build process with the updates for the external dependencies.

If your code depends upon something, then you really need to have control over when it gets updated/modified. Coding in a space where these dependencies can get updated at any time is too painful as you're no doubt finding out. I personally prefer option 1.

Answer 2

I keep them in svn, and manage them as vendor branches. Keeping them loose externally makes it very hard to go back to a previous build, or fix bugs in a previous build (especially if the bug is from a change to the external dependency)

Keeping them in svn has saved me lots of headache, and also allows you to get a new workstation able to work on your codebase quickly.

Answer 3

Have you considered Maven? It's a build system that has excellent support for managing dependencies. For each project you can specify the required dependencies in an xml file as part of that project. The external libraries are held in a dependency repository (in our case Artifactory) this is separate from your version control system and can just be a network drive. It also allows managing different versions of projects.

Answer 4

When I had to do something like this, I added the external source as external, and then applied a patch to it. The patch contains my modifications to the external source. So, I actually only version control my patches. Most of the times this works, if there are no "dramatic" changes in the external code.

Answer 5

I would be careful considering Maven because:

• it is another repository in a system where you already have a repository with your current version control system;
• it (Maven) is based on the only "common version control" every developer have, the file system (which means no metadata, or properties attached to the file, no proper history in term of who modified what and when)

Now when dealing with third-parties, you can consider having them in your version control system, but in a packaged way: that is in a very compact way, with sources and documentations zipped, in order to have the least possible number of files.

That way, you will manage the _**deployment**_ of those (many) third-party libraries easily since the number of files to deploy is low.

Plus, having them under source control allows you to make a branch (say, a 'hack' branch), in which you will stored the packaged (or zipped) version of the hacked library.

What you can store in an external way is the un-zipped, complete set of files representing those libraries since there is no real development on them, or just a punctual hack: normally, your job is not to develop existing libraries, but to use them (even a bit modified) for implementing faster some features of your project.

If you need at some point to compare some hacked version with some official version, you will just pull out from svn the appropriate 'hacked' version number, unzip-it and compare-it with the official (and externally stored) version (with winmerge for instance)