How to test a deployment

Question

We are considering adding another server to development life cycle, so that we can test a deployment.

Some background: We build web applications, using ASP.NET and SQL Server 2005. There are 4 developers in the team and tend to release once every 2 weeks.

This is our current method of deployment: We develop on a Dev server and as each dev case is complete it is added to the Staging server, where it is tested. When we get to the release date, all the cases in a release are deployed from the Staging server to the Live server.

But the problem is that only time when we do the full deployment is when we deploy to Live on the release date – all the deployment to Staging is done case by case. And this has meant that we have been making mistakes or skipping out steps in the live deployment (e.g. forgetting to lock users out during the deployment). What we need is a way to do a dummy run of the live deployment.

What we’re considering is adding another server to the release process, so…

Current server set-up: Dev server -> Staging server -> Live server

Potential server set-up: Dev server -> Staging server -> Beta server (is that the correct name?) -> Live server

That way we could practice each full deployment on the Beta server and draw up a set of steps for the live deployment – and hopefully our live deployments will be smoother. We also plan to give clients access to the Beta server to test things for themselves.

Please let me know what you think. Do you do this or is there another way to test our deployment before the release date?

Answer 1

You are on the right track.

Here's how we work - other people may do things differently (but they can also answer your question and you can decide your own approach).

1) Code is built in dev - this is out uncontrolled area.

2) All database changes are scripted and all .NET code is built into MSI's. We deploy these into Test. They are also stored somewhere special where they can't be messed with. If there are any problems, the testing will find them and we will adjust the scripts / create new MSI's with the fixes.

3) Once testing is complete, the "Pre-Production" environment gets flushed back from live. It's will look like an identical copy of live. The final versions are run into "Pre-Production". The deployment should just work, but this is the opportunity to make sure it does. If it doesn't, the deployment is adjusted and the "Pre-Production" environment is flushed back from live, so we know we're testing it against an exact copy (no point testing it against the one that you've already messed with!)

4) If the release works (and you would usually perform some testing to check all of your components) it's ready to be run in live.

It helps if you make your database scripts re-runnable. Detect whether the change already exists before doing it again, in case the scripts are run more than once for any reason. For example, if you add a configuration value to a table, check to make sure you haven't already done it, else you could add it many times and cock up your data.