How do you organize your Software Development Infrastructure?

Question

We are working on improving our Software Development environment in a bid to produce quality software. My question is how should our infrastructure be set up such that we have,

1. Source Control - for revision control & source sharing
2. Continuous Integration - for continuous builds & pushing code to the staging sever
3. Staging - hosting of applications internally

We currently have only one server which I suspect is not good practice. I would welcome suggestions regarding how we can distribute these features across multiple machines.

Answer 1

I'm working with subverion for source-control (has plugins to most known IDE)

Cruse Control will give you continue integration

and Staging we use our own written scripts...

(and Everything is for free...)

Answer 2

• Your source control repository keeps all your work. You want to run it on a dedicated machine that runs nothing else and ensure its availability as much as possible. (As a side effect, this also gives you the ability to lock down and secure access to it, if needed)

• Your CI should run on a dedicated machine as well, mostly because you might be running multiple builds and that takes resources. Also, never install your application on the CI server. Keep a regular image of the CI machine setup, so that if anything happens to it you could bring it back up as soon as possible with all the necessary tools already installed.

  Btw, it's a good practice to run the CI jobs under a dedicated account that nobody uses for development. (And again, you might want to secure that machine, if it's producing your final RTM bits - you might not want anybody being able to build binaries signed with your company cert, for example)

• Staging also goes to its own machine. After all, you want to ensure that the staging environment is as close to the production one as possible. Keeping a regularly updated image of the staging machine setup (without your app, of course). Don't ever install dev tools on the staging machine, unless your app will use these in production.

  It is also good to have actually two staging servers - INT where you deploy daily builds and PPE where you deploy stable builds, that will be moved to production (like beta, RC or RTM builds).

Answer 3

Actual products depends on the environment you work in.

1: Source Control: Subversion or Git are probably the most used ones. This software should probably go on its own machine. This machine should be backed up regularly as it contains most of you (code) assets.

2: You will find plenty of tools including free ones. TeamCity, CruiseControl (.NET), Hudson, Bamboo to name a few. Again, this would probably go on a machine of its own and preferably a fast one. Make sure you have some kind of standard build framework like MSbuild, NAnt, Maven, Ant to name a few. On here you could do as well some code metrics checks and run your unit tests.

3: Staging depends on the environment you work in. In Java, I guess Maven with a central repository (for example Nexus) could be employed.

Answer 4

We use a Developer Machine that builds and runs everything in the project for each developer that is part of the team. So all developers have a working (complete) system on their box. We have a build server that either does nightly builds or continues integration. The build server runs all Unit Tests - always. We have our source control (SVN or TFS) on a seperate server as well (shared between projects).

Then we have a Developement Environment. Usually just a simplification of the deployment of the solution (so no load balancing and stuff). This is the first real distributed integration of the solution. Its also a playground for the development team to try out things.

We have a Test Environment. Usually similar deployment to the Develepment Environment. This is the place that is used by the Test Team to run their tests. Any defects are reported to a central repository (either a SharePoint List or TFS Work-Item) and fixed by the development team.

Then there is an acceptance environment that matches the production environment as closely as possible. This environment is used for pre-production tests and go/no-go decisions by the customer (acceptance of the new version).

And there is the Production Environment which is never under the control of the development team (Development and Test usually are) and runs the final bits.

Each release/version is packaged and moved through the environments, never changing anything to the package. So if the test team finds bugs, the developers fix them and make a new package that contains these fixes and start at the Development Environment.

So its cheapest to catch bugs early and that is why Unit Testing is important but also having a good test team ;-)

Most of these environments are build using virtual servers. Database servers (SQL-Server) /clusters as well as BizTalk (messaging) are usually not virtual in a production environment.

Hope it helps.

Marc Jacobi