Lean Software (Webapplication) Release Engineering

Question

Hi,

I'm looking to re-organize the way we release our internal software. All of the code (PHP webapps, some Java apps and Perl scripts) is checked into Subversion repositories but there are no branches or tags, everything is checked into trunk (only around 1-3 devs per app). On the production linux servers, the software is just directly run from a working svn copy (actually most of the changes happen there as well).

Since we have a lot of small apps and release very often small changes to the running system, I'm looking for a very lean or transparent way to do some release engineering and to clean up this mess abit.

Are there any tools out there that may help me to do so in a heterogenous environment (language-wise) like that? Or has anyone an idea how to do this in a proper way?

Otherwise I'd thought of writing some release (shell) scripts that automatically create subversion tags from trunk and then do a checkout of the corresponding tag to the production servers. But that sounds kinda hack'ish as well to me.

Thanks,

Haes.

Answer 1

Some Continuous Integration Servers do this sort of thing, Hudson, for example, has subversion integration. It can tag, run test, and deploy for you.

Answer 2

Use tags and branches; make it a part of the development cycle. When you update that "stable-1.0" branch, have tested the change(s) and tagged it "release-1.0.5", you simply do "svn switch" on the server to the new tag. Didn't work, despite having tested it? Switch back, and figure out what's wrong.

But beware, branching in subversion can be a pain, at least pre version 1.5. If you or your developers are not experienced with branches, expect a bit of hassle and/or mistakes in the beginning. But as long as you've committed no code should be lost (at worst simply difficult to merge).

Your developers really should learn how to use branching; it can be very useful for a variety of purposes (not just for release engineering).

Do not automatically switch over code on your production servers; somebody might accidentally hit the wrong button. Production updates should always be done with care. Scripts for adding new tags is, imho, unnecessary due to the simplicity of it, but your mileage may vary.

One last thing, don't allow anyone to have changes on your production server. It might cause conflicts, and those tend to take time to resolve. Not to mention, it destroys your ability to reproduce a given release on different workstations (works fine here! why not on the server? hmm).

Answer 3

Continuous Integration is definitely the way to go - any CI (even minimalist batch files) is better than none - but it'll only be as good as the policies you have in place. Since your files don't really end up as a 'binary' or 'distributable', marking a release might merely require only that you tag the repository, or even just stash the Subversion revision number somewhere. The important policy that you need is that any release can be reconstructed whenever you need it - so you can compare current and previous releases, or go back to an older release if something goes wrong. Don't worry about the 'overhead' of creating tags in svn - that's very efficient.

A release script that does the subversion tag sounds fine. A CI implementation (I'd recommend CruiseControl since it's ideal for heterogeneous work, although heterogeneity requires a bit more configuration overhead) is great, since you can automatically kick the process off on a subversion checkin, and run automated tests that determine whether it's good enough to tag or not.

I'd definitely not auto-deploy to a release server. A 'staging area' (call it 'nightly build', 'beta test', whatever) would be better. Let your users bang away on that before you decide it's good enough to roll out onto the production servers. And, as long as you've got the policy in place of being able to rollback to an earlier version, you've mitigated the possibility of a bad roll-out.

The auto-checkout onto production servers is the only 'hackish' part - an automated checkout, test, tag, beta deploy is slick enough. Rolling-out to production shouldn't have an easy button, though.

Answer 4

i'd use Hudson. in addition to fetching from and tagging in svn (ref sblundy), it can be useful in release management with the proper plugins. f.ex., you could try a plugin to "promote" the builds you deploy to production, and keep a list of both the promoted builds themselves and a change/commit log for the various versions.