How to migrate from "Arcane Integration" to Continuous Integration?

Question

Right now a project I'm working on has reached a level of complexity that requires more than a few steps (actually its become arcane!) to produce a complete/usable product. And unfortunately we didn't start out with a Continuos Integration mindset, so as you can imagine its kind of painful at times, and at others I can easily waste half a day trying to get a clean/tested build.

Anyways as any HUGE project it consists of many components in many different languages (not only enterprise style Java or C# for example), as well as many graphical, and textual resources. Now the problem is that when I look for Continuos Integration, I always find best practices and techniques that assume one is starting a new project, from the ground up. However this isn't a new project, so I was wondering what are some good resources to proactively start migrating from Arcane Integration towards Continuos Integration :)

Thanks in advance!

Answer 1

I would guess that migrating isn't really an option--Half-ass solutions will only make it worse.

My approach would be to take one creative engineer who understands the build process, sit him down and say "Fix this". Give him a week or two.

The end goal would be a process that runs beginning to end with a single make command.

I also recommend an automated "Setup" procedure where you simply do a checkout and run a batch file from a network share to install and build all your tools. The amount of time this will save overall is staggering if you bring in new programmers. Most projects take one to three days to get set up on a new computer--and it's always the "new" programmer who doesn't know what's going on doing the installs on his own system...

Answer 2

The same way you eat an elephant (one bite at a time) ;-) Continuous integration requires an automated build. Start with that. Automate the building of each piece. Ant or NAnt is a great way to do this. Have each component's construction be a NAnt task. Then your entire system build can aggregate those individual tasks.

From there, you can add tasks for deployment, for unit testing, etc. If you want to use a CI technology, you can wire it up to your NAnt build.

Answer 3

In short: Incrementally

Choose a framework that will work across the diverse range of projects.

One by one, add components to the framework.

If you are not familiar with the framework, tackle a couple of the easier components first, to reduce risk of screwing up.

If you do understand the framework, tackle some of the more difficult and/or commonly built components first, so your team (and management) will appreciate the benefits early, and support the effort more.

Be sure to have a plan to include all of your components, because that's when the full benefit will be realized.

Bring your team with you; make sure you have consensus that this is going to be valuable, or people won't maintain it as the components change.

Answer 4

Here it is in two simple (hah) steps.

1. Go for the repeatable build:
   1. Use source control, get all code checked in.
   2. Establish and document all tools used to build (mainly, which compiler version). Have a repeatable deployment and set up process for these tools.
   3. Establish and document clearly any resources which are necessary to build, but are not checked in (third party installations, service packs, etc). Have a repeatable deployment and set up process for these dependencies.
2. Before commiting to source control, developers must
   1. update their working copy
   2. successfully build
   3. run and pass automated tests

These steps can be done 1 at a time, sort of a path to follow. You'll get benefits at each stage. For example, if you aren't using source control at all, just getting the code into source control (without anything else) is a big step forward. Also, if there are no automated tests, then developers can't run them - but they can still get the prior commits and get the compiler to check their work.

If you can do all of these, you'll get to a nice sane place.

The goals are repeatable build processes and developers that are plugged in to how their changes affect the build and other developers.

Then you can reap the bonuses by establishing higher compliance:

• Developers establish a frequent commit habit. Code that is in the working copy should never be more than 1 day old.
• Automated build process monitors source control for check-ins and gets the results to a place where the users can accept them (such as a test environment, a preview website, or even simply placing an .exe where the user can find it).

Answer 5

I would start by first writing down all the steps it takes you to do the build and test manually. After that you at least have a guide for doing it the old way, and writing things down gives you the chance to look at it as a complete process.

Then look for parts to script.

Ideally you want to trigger a build and test from a code commit and only rebuild and retest the changed parts, with perhaps a full build and test nightly or weekly. You'll need log files or database entries and reports on the build success or lack of it.

You'll want to search out and evaluate pre-built products and open-source build-your-own kits. You can certainly write all the scripting and reporting yourself, but it will take a while and you'll probably end up with a just barely good enough reporting system since your job is coding the product, not coding the build system. :-)