SVN: Release branch headaches, how to merge in website revisions as and when cleared to go live?

Question

I need a sanity check here if we can, any ideas on correcting/changing the following are very welcome! We've been getting ourselves in knots of late with our SVN and are trying to correct it by putting a Trunk/Release system in place.

We have a large website that we develop on and we store it all in SVN. Heres what we had in mind:

• We have trunk and a release branch
• All work gets checked into Trunk.
• When a feature is deemed ready for the next release it is merged into a Release branch.
• We only have one release branch and just tag "Latest" when we do a push to live
• We hope to be able to get all the files changed from Latest to Head to give us a zip that we can upload (any ideas on an easy way to do this via scripting?)

So we set all this up and where very happy with ourselves. Except its not working and heres why.

We work on lots a different features/fixes/problems at once and they don't all get nicely checked in feature complete (but always working at least). Then sometimes you have to wait for Clients to sign off. As a result you end up with revisions which are "ready for live" being scattered with ones which are "still being worked on" in trunk. That means that the completed revisions are not getting merged in sequentially but out of order. I thought SVN could handle this, clever little thing it is, but apparently not.

Heres an example:

• Pete changes some CSS to make a new button look pretty (Revision 1)
• Dave add some CSS to the bottom of the same CSS file as Pete's for a new feature (Revision 2)
• Dave's mod gets the nod so he merges it into Release and commits it with a log message mentioning revision number and bug tracking id.
• Pete adds more buttons to finish this mod, no CSS changes here though (Revision 3)
• Pete then merges his mods (Revision 1 and 3) into the Head of Release (which has Daves merge in it) but this over-writes Daves CSS additions which now dissapear completely.

This leads to the site being broken and the Release branch being pretty much useless.

So we tried some other ideas like reverting Release back to "Latest" and then just merging in all the Revisions 1,2 and 3 in order. This worked fine until we had Revision 4 which was not ready for live and Revision 5 which was. Suddenly we are getting ourselves in knots again with exactly the same problem!

Ok so take three. Revert to Latest, merge in Revision 5, then do any update back to Head. Tree conflicts galore! So thats a no no.

I cracked in the end and built it all up manaually but its not something I want to do regular, ideally I want to script our deployment but can't while Release is in such a mess.

HELP! What the heck are we doing wrong? I can't seem to find any solutions to this problem of wanting different none sequential Revisions in Release. If its not possible thats fine but how the heck are we meant to get stuff live easily. We can't branch for every single change, the site takes 30 minutes+ to check out it would take too long.

Side note, we are using TortoiseSVN so can we keep command line examples to a minimum in any answers?

Latest version of TSVN and SVN Version 1.6 so we have the funky merge tracking etc.

EDIT: An excellent blog post which deals with the dev/release cycle (although using GIT but still relivant) thought everyone would like to read it if they found this question interesting. (http://nvie.com/git-model)

EDIT 2: I wrote a blog post on how to show which branch you are working on in your website which others have asked me about (http://www.offroadcode.com/2010/5/14/which-svn-branch-are-you-working-on.aspx). Hope that helps. In the meantime we are looking at Kiln and hoping to make the switch next month (gulp!)

Answer 1

Hi,

Usually, people would develop on trunk, like you do, and at one point in time decide to release thus branching with a "release_something" name. It is not an on-going process of merging things in the release as much as possible but more a tag and freeze process :-). Then the work done on the release branch would consist mainly of bug fixes, merged as soon as possible in the trunk as well: with that kind of process, it is much easier.

From what you say you want to achieve, it seems you are right: the best solution would be to have one branch for each feature, then merge them one by one in the trunk and selectively in the release branch. Subversion does not seem to be the best tool for that kind of development. If you were using a DVCS like hg or git, you could have one repository for each developer, ask them to merge/push as soon as possible their changesets into the trunk, in order to test the maximum of features integrated together and find potential problems. But you will also have another repository, called "release", maintained by only one person, who will pull selectively such and such feature/patch from your developers repositories. In hg for example, which I know better, when you pull changes that have not yet been merged (e.g., from a dev repo to the release repo), mutiple "heads" are created, i.e., some kind of anonymous branches, and you can decide change by change, feature by feature, if you want to merge these "heads" into the main branch of not.

I believe because of this flexibility, and because DVCS tools are able to handle branches and merges much better than centralized tools, it would suit your needs much better.

Hope it'll help.

Cheers,
Christophe.

= They always say that time changes things, but you actually have =
= to change them yourself. --Andy Warhol =

Answer 2

What the heck are we doing wrong?

You are doing several development efforts on one branch, and those efforts should not be done at the same time (because their lifecycle is different, i.e. they won't be released at the same time)

You should define a branch per development effort (and not per developer), and merge those branches in a pre-release branch (to integrate all the approved developments for release), with a final merge to Release when all developments have been validated together.
If each development is fairly sequential per developer, that mean each developer can keep one workspace per general task, and checkout only the relevant workspace for that set of tasks: the number of files actually changing would be small, reducing drastically the checkout time.

See:

• purpose of branching
• branches and SVN
• merge workflow (to minimize conflicts during merges)

Answer 3

Let's try the scenario that you describe, and see what actually happens (I'm using the command line for obvious reasons, but it works the same in TortoiseSVN):

# let's create a simple repository for testing and add some baseline data:
$ svnadmin create test

$ svn import baseline file:///path/to/repo/test/trunk -m "Import baseline"
Adding         baseline/other.txt
Adding         baseline/css.txt

Committed revision 1.

# now create a branch:
$ svn copy file:///path/to/repo/test/trunk file:///path/to/repo/test/branches/r1 -m "branched" --parents

Committed revision 2.

# Pete changes some CSS to make a new button look pretty (Revision 3)
$ svn co file:///path/to/repo/test/trunk Pete-trunk
A    Pete-trunk/other.txt
A    Pete-trunk/css.txt
Checked out revision 2.
$ svn diff Pete-trunk/
Index: Pete-trunk/css.txt
===================================================================
--- Pete-trunk/css.txt  (revision 2)
+++ Pete-trunk/css.txt  (working copy)
@@ -1,3 +1,3 @@
 This is a test
-more lines
+more lines Pete's change
 even more
$ svn commit Pete-trunk/ -m "Pete's first change"
Sending        Pete-trunk/css.txt
Transmitting file data .
Committed revision 3.

# meanwhile, Dave add some CSS to the bottom of the same CSS file as Pete's for a new feature (Revision 4)
$ svn co file:///path/to/repo/test/trunk Dave-trunk
A    Dave-trunk/other.txt
A    Dave-trunk/css.txt
Checked out revision 3.
$ svn diff Dave-trunk/Index: Dave-trunk/css.txt
===================================================================
--- Dave-trunk/css.txt  (revision 3)
+++ Dave-trunk/css.txt  (working copy)
@@ -1,3 +1,4 @@
 This is a test
 more lines Pete's change
 even more
+Dave's change
$ svn commit Dave-trunk/ -m "Dave's change"Sending        Dave-trunk/css.txt
Transmitting file data .
Committed revision 4.

# Dave's mod gets the nod so he merges it into Release and commits it (Revision 5)...
$ svn co file:///path/to/repo/test/branches/r1 Dave-branch
A    Dave-branch/other.txt
A    Dave-branch/css.txt
Checked out revision 4.
$ svn merge -c4 file:///path/to/repo/test/trunk Dave-branch
--- Merging r4 into 'Dave-branch':
U    Dave-branch/css.txt
$ svn diff Dave-branch/

Property changes on: Dave-branch
___________________________________________________________________
Added: svn:mergeinfo
   Merged /trunk:r4

Index: Dave-branch/css.txt
===================================================================
--- Dave-branch/css.txt (revision 4)
+++ Dave-branch/css.txt (working copy)
@@ -1,3 +1,4 @@
 This is a test
 more lines
 even more
+Dave's change
$ svn commit Dave-branch/ -m "Merged Dave's change to release"
Sending        Dave-branch
Sending        Dave-branch/css.txt
Transmitting file data .
Committed revision 5.

# Pete adds more buttons to finish this mod, no CSS changes here though (Revision 6)
$ svn diff Pete-trunk/Index: Pete-trunk/other.txt
===================================================================
--- Pete-trunk/other.txt    (revision 2)
+++ Pete-trunk/other.txt    (working copy)
@@ -1 +1,2 @@
-another file
\ No newline at end of file
+another file
+Unrelated change
\ No newline at end of file
$ svn commit Pete-trunk/ -m "Pete's second change"Sending        Pete-trunk/other.txt
Transmitting file data .
Committed revision 6.

# Pete then merges his mods (Revision 3 and 6) into the Head of Release (which has Daves merge in it)
$ svn co file:///path/to/repo/test/branches/r1 Pete-branch
A    Pete-branch/other.txt
A    Pete-branch/css.txt
 U   Pete-branch
Checked out revision 6.
$ svn merge -c3,6 file:///path/to/repo/test/trunk Pete-branch
--- Merging r3 into 'Pete-branch':
U    Pete-branch/css.txt
--- Merging r6 into 'Pete-branch':
U    Pete-branch/other.txt
$ svn diff Pete-branch/
Property changes on: Pete-branch
___________________________________________________________________
Modified: svn:mergeinfo
   Merged /trunk:r3,6

Index: Pete-branch/other.txt
===================================================================
--- Pete-branch/other.txt   (revision 6)
+++ Pete-branch/other.txt   (working copy)
@@ -1 +1,2 @@
-another file
\ No newline at end of file
+another file
+Unrelated change
\ No newline at end of file
Index: Pete-branch/css.txt
===================================================================
--- Pete-branch/css.txt (revision 6)
+++ Pete-branch/css.txt (working copy)
@@ -1,4 +1,4 @@
 This is a test
-more lines
+more lines Pete's change
 even more
 Dave's change
$ svn commit Pete-branch/ -m "Merged Pete's changes"Sending        Pete-branch
Sending        Pete-branch/css.txt
Sending        Pete-branch/other.txt
Transmitting file data ..
Committed revision 7.

# Now the test: is Dave's change still there? Yes!
$ svn cat file:///path/to/repo/test/branches/r1/css.txt
This is a test
more lines Pete's change
even more
Dave's change

The problem didn't happen, see? And this would still be true if we had done the steps in a different order - try it!

So what can go wrong? Most likely improper resolution of merge conflicts:

For example, maybe Pete checked out his working copy of the branch before Dave merged his changes. He can still merge his changes fine, but when he tries to commit, svn will complain that his working copy is out of date. He has to update his working copy. This will add Dave's change to his working copy; since Dave's change affects the same file that Peter also changed, SVN will then try to merge the changes in the file. If the changes are far apart, this will succeed automatically.

But if the changes are on the same line, or very near to each other, SVN will be unable to merge them, and mark them as conflicted. Now Pete has to resolve the conflict. One option is to use simply his version, discarding Dave's changes. If he does this, Dave's changes will be lost. Instead, he must edit the file (with TortoiseMerge, or possibly by hand), so that it contains both changes.

Answer 4

The best setup I've seen is:

• Trunk always builds and is tested and you "could" go live with it at any time.
• You create tags from the Trunk when ready to release and export from the Tag.
• tags (v.1.0.233, v.1.1.013, etc.)

• Branches are where all of the development is done. Bugs and features. And are merged into the trunk after testing. Possibly in a "Testing" branch.

  root
   - trunk
   - branches
     - bug 1
     - bug 2
     - feature 1
   - tags
     - v.1.0.233
     - v.1.1.013
  

Answer 5

Here's another svn structure to consider, and not unlike the process we use for a similarly large codebase:

root
 - tags
    - project-name-1.0.1
    - project-name-1.0.2
    - ...
 - branches
    - reviewed
    - stable
 - trunk

All commits are done into the trunk, the same way you do now. This is helpful because you catch all conflicts at the time of initial code check-in.

However, instead of merging changes into a stable branch, you merge the changes for the revisions of the approved/verified bug-fixes or features into something like branches/reviewed. It might be good practice to reference the ticket number and original revision in the svn:log. Then once you are ready to update the server with approved changes you can do an automatic merge to the stable branch, since any weird conflicts would be caught earlier in the process.

You may run into some conflicts with the second step, since you're applying the revisions out of order. Subversion should be able to handle this, but if you run into any problems a strategy to work around them would be:

Find the revision in the trunk that only has approved changes, then

1. selectively merge the subsequent approved changesets in order, or
2. Merge all revisions from r{all-approved} to CURRENT, then reverse merge each subsequent unapproved changesets

Both of these will be a pain, but again, this shouldn't really happen very often.

One last thing: most of our developers use TortoiseSVN like you, but you can save a lot of time doing these sorts of merge operations with bash scripts or even perl/php scripts. You might consider setting up a unix box or virtual machine for this sole purpose.