Improving Your Build Process

Question

Or, actually establishing a build process when there isn't much of one in place to begin with.

Currently, that's pretty much the situation my group faces. We do web-app development primarily (but no desktop development at this time). Software deployments are ugly and unwieldy even with our modest apps, and we've had far too many issues crop up in the two years I have been a part of this team (and company). It's past time to do something about that, and the upshot is that we'll be able to kill two Joel Test birds with one stone (daily builds and one-step builds, neither of which exists in any form whatsoever).

What I'm after here is some general insight on the kinds of things I need to be doing or thinking about, from people who have been in software development for longer than I have and also have bigger brains. I'm confident that will be most of the people currently posting in the beta.

Relevant Tools: Visual Build Source Safe 6.0 (I know, but I can't do anything about whether or not we use Source Safe at this time. That might be the next battle I fight.)

Tentatively, I've got a Visual Build project that does this:

1. Get source and place in local directory, including necessary DLLs needed for project.
2. Get config files and rename as needed (we're storing them in a special sub directory that isn't part of the actual application, and they are named according to use).
3. Build using Visual Studio
4. Precompile using command line, copying into what will be a "build" directory
5. Copy to destination.
6. Get any necessary additional resources - mostly things like documents, images, and reports that are associated with the project (and put into directory from step 5). There's a lot of this stuff, and I didn't want to include it previously. However, I'm going to only copy changed items, so maybe it's irrelevant. I wasn't sure whether I really wanted to include this stuff in earlier steps.

I still need to coax some logging out of Visual Build for all of this, but I'm not at a point where I need to do that yet.

Does anyone have any advice or suggestions to make? We're not currently using a Deployment Project, I'll note. It would remove some of the steps necessary in this build I presume (like web.config swapping).

Answer 1

I have a set of Powershell scripts that do all of this for me.

Script 1: Build - this one is simple, it is mostly handled by a call to msbuild, and also it creates my database scripts.

Script 2: Package - This one takes various arguments to package a release for various environments, such as test, and subsets of the production environment, which consists of many machines.

Script 3: Deploy - This is run on each individual machine from within the folder created by the Package script (the Deploy script is copied in as a part of packaging)

From the deploy script, I do sanity checks on things like the machine name so things don't accidentally get deployed to the wrong place.

For web.config files, I use the

<appSettings file="Local.config">

feature to have overrides that are already on the production machines, and they are read-only so they don't accidentally get written over. The Local.config files are not checked in, and I don't have to do any file switching at build time.

[Edit] The equivalent of appSettings file= for a config section is configSource="Local.config"

Answer 2

I've only worked on a couple of .Net projects (I've done mostly Java) but one thing I would recommend is using a tool like NAnt. I have a real problem with coupling my build to the IDE, it ends up making it a real pain to set up build servers down the road since you have to go do a full VS install on any box that you want to build from in the future.

That being said, any automated build is better than no automated build.

Answer 3

We switched from using a perl script to MSBuild two years ago and haven't looked back. Building visual studio solutions can be done by just specifying them in the main xml file.

For anything more complicated (getting your source code, executing unit tests, building install packages, deploying web sites) you can just create a new class in .net deriving from Task that overrides the Execute function, and then reference this from your build xml file.

There is a pretty good introduction here: introduction

Answer 4

Our build process is a bunch of homegrown Perl scripts that have evolved over a decade or so, nothing fancy but it gets the job done. One script gets the latest source code, another builds it, a third stages it to a network location. We do desktop application development so our staging process also builds install packages for testing and eventually shipping to customers.

I suggest you break it down to individual steps because there will be times when you want to rebuild but not get latest, or maybe just need to re-stage. Our scripts can also handle building from different branches so consider that also with whatever solution you develop.

Finally we have a dedicated build machine that rebuilds the trunk and maintenance branches every night and sends out an email with any problems or if it completed successfully.

Answer 5

Our build system is a makefile (or two). It has been rather fun getting it working as it needs to run on both windows (as a build task under VS) and under Linux (as a normal "make bla" task). The really fun thing is that the build gets the actual file list from a .csproj file, builds (another) makefile from that, and run that. In the processes the make file actually calls it's self.

If that thought doesn't scare the reader, then (either they are crazy or) they can probably get make + "your favorite string mangler" to work for them.

Answer 6

When taking on a project that has never had an automated build process, it is easier to take it in steps. Do not try to swallow to much at one time, otherwise it can feel overwhelming.

1. First get your code compiling with one step using an automated build program (i.e. nant/msbuild). I am not going to debate which one is better. Find one that feels comfortable to you and use it. Have the build scripts live with the project in source control.
2. Figure out how you want your automated build to be triggered. Whether it is hooking it up to CruiseControl or running a nightly build task using Scheduled Tasks. CruiseControl or TeamCity is probably the best choice for this, because they include a lot of tools you can use to make this step easier. CruiseControl is free and TeamCity is free to a point, where you might have to pay for it depending on how big the project is.
3. Ok, by this point you will be pretty comfortable with the tools. Now you are ready to add more tasks based on what you want to do for testing, deployment, and etc...

Hope this helps.

Answer 7

Some good stuff so far, thanks all for replying. I'm looking into that MSBuild intro right now. I'm not obliged to use Visual Build (and have a few reservations about using it). I've heard of Cruise Control and I'll have to research it some later, but I don't think we're quite ready for it. From a certain perspective, we're kind of backwards/primitive when it comes to this stuff.

Rob Allen, that link you provided was broken. I did some searching on various build related phrases but didn't find a whole lot, so I'd love to read whatever thread you were trying to link.

Answer 8

One thing I would suggest ensure your build script (and installer project, if relevant in your case) is in source control. I tend to have a very simple script that just checks out\gets latest the "main" build script then launches it.

I say this b/c I see teams just running the latest version of the build script on the server but either never putting it in source control or when they do they only check it in on a random basis. If you make the build process to "get" from source control it will force you to keep the latest and greatest build script in there.

Answer 9

We use UppercuT. UppercuT uses NAnt to build and it is extremely easy to use.

http://code.google.com/p/uppercut/

Some good explanations here: UppercuT