What are the best practices to use to pass a web application from development staging to production?

Question

Our Development environment has many layers and is complicated to replicate or even backup effectively. Basically the File system (ie. /usr/appdir/webapp...) has other applications serving our web application, those application we update doing svn updates from their repository.

The use of the web application itself (as a user) will affect both the file system and the database. So backing up the system is a matter of having a copy of both the file system and database (mysqldump) at the same point in time. One of the two by itself will not be a complete backup since the application is very dynamic itself.

When we deploy an webapp to staging for one of our customers to test and enter data, then we have an environment that now is difficult to sync back from our development environment, or even to take it to production. Since we will make change request from the customer in development, but the customer itself will make changes in staging.

At this moment we are using freezing periods, where we ask our customers to make changes to development environments or even directly to production (before going live completely).

I am wondering if their is a best practice on how to have a effective process to pass from dev --> staging --> production? Or if you might have some pointers.

Answer 1

I typically don't make any changes in the QA/staging environment. Any change gets made in the development environment and then pushed to both QA/staging and production. Normally, I don't use live data in QA/staging either if I can avoid it. In cases where I do have live data in QA/staging, I'll use something like SQL Data Compare (from Red Gate) to manage the migration of new data from the QA/staging environment to production.

Also, I maintain separate configurations for dev/QA/production in source code control. It's a bit of a pain to have to remember to make changes in all configs when the change affects them all, but there are enough differences that it is easier to do this than to update the config for QA/production when the app gets published/installed.

Answer 2

From the sound of it, your problem is complicated by the nature of your application.

Firstly, does your application make a distinction between files that are part of the application, and files that are mere data? If not, unless you have a very compelling reason for not doing so, a change of that nature should make version control of your filesystem much simpler - then it would make sense to have the application parts under version control, but not the data parts.

Secondly, I have always found keeping filesystem data in sync with database data a pain. One solution (which may be a more extensive refactoring) is to make one generate-able from the other. Could you somehow store the filesystem stuff in your database linked to the relevant data, that could populate the filesystem from a script? If your files are too big, in many cases having representative files for the purposes of development and testing will suffice.

Alternatively, you could have a separate repository for data, which can take a valid snapshot of a set of data (both database and filesystem) which can be stored and managed separately to your application code (although this approach will bring other problems...)

If the design issues here are immutable, I can't think of a better idea than your freezing periods.