Production Release Checklist

Question

When releasing a new build to production, what is the general preperation/checklist you go through to make sure your build isn't going to flop? What I am looking for is something like

Technology: ASP.NET & SQL Server

1. Backup Production Database
2. Restore Production Database to UAT
3. Run database alter scripts
4. Grab latest buid from Cruise Control
5. Push build to UAT
6. Run automated tests with Tool or test manually by hand
7. Push UAT to Production

Answer 1

Only really obvious step you're missing is to run checkouts in Production ;)

If you have the application running on in a distributed environment, you may have to...

• Take box A out of load balancer, dns, etc.
• Move code to box A.
• Run Checkouts.
• Put Box A back into load balancer, dns, etc.
• Rinse Repeat.

EDIT w/ more details

Here's how we do it at Artie Ziffcorp. We put a change ticket in the change request log, inform Net Ops that we are doing a change (so they know that any errors on the monitoring tools are bogus), build team clean builds from repository moves, assets to box, biz checks out box, dev team does triage as necessary, recycle jvms/iis/your app server here. When the smoke clears put the box back in the dns/load balancer and call up Net Ops and tell them the change is done.

Party like your dev team just had a release. ;)

Most of our DB changes are already in place come release time, the views just point to different tables.

Id say on an average release, roster is about 2 engineers (to check out the production box), the dev team (usually 5-10), test team (2-5), build team (1 rep from that department), and 2-4 project managers/business types.

35k person company and about 10% of them are strictly IT.

Hope this provides better insight, it is by no means the only way to do a release.

Connecting Visuals to Gameplay (WARNING 7MEGS) Go to page 36. Talks about Valve's Shipping Machine!

[1]: http:// www.valvesoftware.com/publications/2008/MIGS08_ConnectingVisualsToGameplay.pdf

Answer 2

We do it like this (and I'm not evangelising this, but our deployment is a lot better than it was).

1. Build and publish latest release version by switching my build configuration to release - every config setting that needs to change from development and staging ready for production is governed by our web.config which auto-switches based on the build configuration.
2. Perform database diff against our development database and a script of our live database taken from our last deployment - we use Apex SQL Diff and Script for this.
3. Create a staging database from the database generation script created at our last release.
4. Run the database diff script on staging to upgrade it and install the new published build of our app from step 1 (staging and live have the exact same server configuration)
5. Perform testing on staging.
6. If tests pass, then prepare to go live:
7. Backup the code and sql database on live.
8. Deploy the new build and run the db upgrade script.
9. Perform additional testing on live to validate it is working.
10. Roll-back immediately on any unexpected error.

We're getting there, we need to automate this more so are working with MSBuild to automate builds, package and move the code to staging areas. The creation of database diffs and comparison to the last deployed schema/data is performed by SQL Diff/Script which can be run from a command line and integrate with subversion. Backing up, upgrading the db and deploying the new code can all be scripted including rolling back. We'd like to move to something like this:

1. Run job to build, create db upgrade script, create staging, run db upgrade, deploy code to staging, run tests and check test result.
2. On test pass, backup live, take web site offline, deploy new code and upgrade database, run scripts.
3. Rollback on test fail.

One-click deployment with fail-back? That's our ultimate goal.