How to Manage Web Development?

Question

My question is somewhat related to this question.

What are good ways to set up a reasonably efficient environment for web development? I am using Subversion, and have two servers, a live one, and a dev site that is pretty much a mirror for the code. Up to now, I have been coding directly on the dev site, but I'm now thinking I should pull back and just use it for testing, with each developer having his own version of the site.

This does mean having to set up local versions of the database etc. for each user though, and has complications of their own.

Does anyone know of a good way to manage this? Perhaps setting up virtual machines and just cloning them to each person, using VMWare perhaps?

One slight problem is that some of the code depends on the URL (specifically the subdomain), which means you couldn't just run things on localhost (at least without changing the code I guess, which is a possibility).

In any case, I am quite sure issues like this have cropped up again and again, so I would be very interested to learn how other people deal with these issues.

I should probably mention that most of the code I write for the web is PHP, though with some Javascript as well. One big benefit of switching to local machines is that I could use a proper debugging environment for PHP, as I am starting to get very frustrated with just echoing data structures to the browser as a way to fix issues.

Answer 1

The best setups I have seen are ones where you have a production, testing, and then development is done on the developers machines like you are saying. However with one difference that most people will agree on. Have one database server that serves all the developers local instances, this is important because it keeps from the database getting out of sync on each of the developers machine and then having to deal with merges.

Then you just specify one person, usually the lead, who does the database development and data layer of the application. This way you don't need to worry about all the problems associated with multiple versions of a database, and since it is only structurally changed by one person which lets the business and ui layers be focused on by the other developers.

Personally this has worked great for me ranging from 2 person to 25 person teams.

Answer 2

To get round your domain problem a developer could add the domain to their hosts file and map it to their local machine.

I'm sure there must be an equivalent for *nix

Answer 3

A scripting language like Perl, Python, Ruby (see Capistrano), or even PHP itself (see Phing) is your friend in this situation. Every time you repeat something 2-3 times, encode the task in a script, and you can run it any time you need to set up a new project or do something similarly repetitive.

You can run everything on virtual hosts on local dev machines by editing the /etc/hosts file (the Windows equivalent is C:\WINDOWS\system32\drivers\etc\hosts). You should have a config variable which allows you to change the hostname for dev, testing, and production environments. There's no real alternative to this, except to run everything out of subfolders on localhost (yuck).

There are two common ways to manage and share changes to the database during development - the easiest and cleanest is to reinstall the entire DB and load in all the default data from fixture files every time the schema changes. The alternative is to use a migration script which versions the schema, and performs "alter table" or "rename" modifications to the schema at every version change. The advantage of this is that you can usually script the reverse migration which allows you to rollback the schema to the previous state. Once your database goes into production, you will have no alternative - you'll need to use some form of migrations.

Some teams prefer to share databases using SQL dumps - the same concepts apply, just that instead of modifying the schema in task code, the raw SQL for the DB is kept in files and plugged in and out when needed. Obviously, this becomes impractical on larger databases.

Answer 4

My approach is to have a script which initializes my database by creating the necessary tables and inserting whatever rows are necessary for the site to function. Once the database has actual data, I'll also have the script load a database dump of the actual data.

At this point, the following steps will create a working dev system:

1. install all necessary packages and programs (which will be necessary no matter what your process is)

2. check the project out from subversion

3. create a database and run the database initialization script

4. configure my web server to behave the way I want (with appropriate permissions, mod_rewrite settings, etc)

After this, I can simply do an svn commit and it will update the code on the server with the subversion post-commit hook, as described at http://subversion.tigris.org/faq.html#website-auto-update

The main hassle is making a change to the database and having to update the other databases as well. While there are tools to make this easier (django-admin.py can do some of this for you) I prefer to do everything manually by distributing a .sql file to do the necessary ALTER and UPDATE statements.

As for your issue of having the URLs hard-coded into your site, I'd advise against that. Perhaps you can do what they recommend in the subversion tutorial with a local settings file for each system. So you check a SETTINGS.FILE.sample into subversion and have everyone make a copy without the .sample extension and use that, storing the URL base there.

@Nick: I've had problems on some projects in which everyone used the same database for testing, particularly in the early parts of a project where the database tables are changing frequently. How much of a problem this is depends on how much each team member is using the same tables.

Answer 5

re: How to set up a test system when the subdomain is used in the development.

I have this, and I've gotten around it by added ".test" domains to my hosts file. e.g.

127.0.0.1      *.metachat.test

• When running live, I go to http://subdomain.metachat.org
• When running on my test system I go to http://subdomain.metachat.test

This works very well.

Answer 6

@Eli,

This is all true about early issues with constant changes in the database. However managing breaking code from DB changes are still much easier than merging changes from separate databases and then managing breaking changes in the code. Plus it will really bite you when two developers have different ideas about what the same database table should be.

That is why I like the rule.

The closer to the hardware you get in an application the less people should be involved.

Basically it means that less and less people should be involved in the process the closer you get to the database. I like to have one person as the database master who implements the requirement changes, then a couple people do the business layer, and then the rest of the team works on the UI layer.

Answer 7

We use 3 tiers - production, staging (test) and dev servers. It's a similar setup to Nicks suggestion, but with development work on a shared server rather than on developers individual workstations. This is a double edged sword though - another developers current work can conflict with your own, but at least you're aware of the impact throughout.

Generally this works pretty well, but we're only a small team of six. I'm sure that teams with more developers would end up treading on each others toes...

Answer 8

I've tried sharing a development database on two separate teams, both with less than desirable results. What I'd say instead is have a file in Version Control which is just a list of things like alter/create statements that change the database schema (or row population queries, for example for a lookup table). This was the developer can keep their local scheme up to date and at the same time work without worrying about stepping on team members toes.

Nick suggests using one database and having a "database guy" but I'm not sure how realistic or productive that is.

Answer 9

We also run 3 tiers at work: Production, Staging, and Dev. Because we're using DB2, we're limited by licensing costs, so all of the devs share a central database. If we weren't limited by that we'd definitely have individual databases per developer.

That aside, I really like how our dev environment is set up. Each developer has a vhost on the development server. When I check out code, I directly edit it in my development branch, which my vhost is serving. When I check it back in, it automatically gets synced to the main central copy of the development server (always contains the most recent checked-in code) and is also synced out to a staging server that is running against the live database.

Obviously, we need to be a bit careful when testing code that does writes on the staging server, but overall we have not had any problems with mangling data.