Web Application Development Process - Version Control, Bug Tracking, Unit Testing, Deployment

Question

Describe the process you use to develop web applications at a not-so-high level, focusing on VC, bug tracking, QA, unit testing, deployment and anything else similar (minus the planning/client communication side of things).

I'm new in this area, so my rough example (read: haven't used this process) is no doubt abit off, so to speak - point out it's flaws so I can learn.

Eg.

1. Create project repository on local SVN server.
2. Create batch/shell scripts for DNS mappings.
3. Check out project, begin work on local working copy.
4. Develop features as branches.
5. Track bugs with Mantis (link commits to bugs through it's SVN integration (no idea if that exists)).
6. Document as you go.
7. Do QA on branch.
8. Merge to trunk when stable.
9. Unit Testing?
10. Commit to repository when feature is implemented and stable.
11. Copy releases to tags in repository. Eg. /project/tags/rel-123/
12. Use Phing to upload to staging server. (Could someone please clarify exactly what a staging server is used for beyond 'testing'?)
13. Use Phing to prep live site for update, set up DB/deploy, etc.

Answer 1

Very roughly:

1. Create repository in SVN
2. Checking local working copy to developer environment
3. Update/commit changes frequently
4. Deploy to stage from SVN trunk using custom deploy script
5. QA tests on stage, reports bugs in Mantis
6. Developers fix bugs, mark as resolved
7. Re-deploy to stage
8. QA tests bugs, closes if fixed
9. QA is finished, do regression testing
10. Deploy to production using custom deploy script
11. Do a little dance

We also create branches for future versions or features. These eventually get merged into the trunk.

We keep our db structures synchronized with a custom db comparison tool that is executed during the deploys.

Answer 2

1. Create/checkout HEAD version ("main branch")
2. Develop code and sync with the main branch -at least- daily
3. After development is done, write and run unit tests
4. Go through code review and submit code/changes to the main branch
5. Let continuous builder run all unit tests and system/integration tests on main branch
6. When ready, cherry pick revisions and integrate them to the QA branch
7. Run system and integration tests, fix reported bugs, or rollback as necessary; this repeats steps 4-7
8. After QA signoff, integrate QA change to release branch
9. Run unit tests, system/integration tests on release branch
10. Deploy to production and run sanity tests.

A staging server is a copy of your production environment that is as up-to-date as possible. On my current project, we're able to keep each release independent from each other, so our "staging server" is our production server, just accessed from a different url.

Notes and discreprencies:

All of the steps have some variation depending on the size of your project. The larger your project, the better the benefit from cherry picking and environment separation. In smaller projects, these can just be time sinks and are often ignored or bypassed.

To clarify, there is a Development stack, QA stack, and Staging stack. Depending on your project size, QA might be staging, Production might be staging, or some combination thereof. The separation of the Dev and QA stacks is the most important one.

In the steps above, I'm assuming both code and relevant data is versioned or tracked. Having a release and build process that takes control data into account makes a release very, very easy.

In a small-medium sized project, there may or may not be a release branch; it depends on the frequency of code change. Also, depending on the frequency of code change and size of your project, you may integrate the full QA branch to the Release branch, or cherry pick specific revisions to integrate to the release branch.

FWIW, I've found "migration scripts" to be of little value. They're always a one-off script with little reuse and make rollbacks a pain in the ass. Its much easier, I would argue better, to have the application backwards-compatible. After a few releases (when a rollback is a laughable), data cleanup should be done, if necessary.