views:

746

answers:

11

Perhaps the biggest risk in pushing new functionality to live lies with the database modifications required by the new code. In Rails, I believe they have 'migrations', in which you can programmatically make changes to your development host, and then make the same changes live along with the code that uses the revised schema. And roll both back if needs be, in a synchronised fashion.

Has anyone come across a similar toolset for PHP/MySQL? Would love to hear about it, or any programmatic or process solutions to help make this less risky...

+2  A: 

Symfony has a plugin called sfMigrationsLight that handles basic migrations. CakePHP also has migrations.

For whatever reason, migration support has never really been a high priority for most of the PHP frameworks and ORMs out there.

jcoby
+4  A: 

I've never come across a tool that would do the job. Instead I've used individual files, numbered so that I know which order to run them: essentially, a manual version of Rails migrations, but without the rollback.

Here's the sort of thing I'm talking about:

000-clean.sql         # wipe out everything in the DB
001-schema.sql        # create the initial DB objects
002-fk.sql            # apply referential integrity (simple if kept separate)
003-reference-pop.sql # populate reference data
004-release-pop.sql   # populate release data
005-add-new-table.sql # modification
006-rename-table.sql  # another modification...

I've never actually run into any problems doing this, but it's not very elegant. It's up to you to track which scripts need to run for a given update (a smarter numbering scheme could help). It also works fine with source control.

Dealing with surrogate key values (from autonumber columns) can be a pain, since the production database will likely have different values than the development DB. So, I try never to reference a literal surrogate key value in any of my modification scripts if at all possible.

yukondude
+4  A: 

I don't trust programmatic migrations. If it's a simple change, such as adding a NULLable column, I'll just add it directly to the live server. If it's more complex or requires data changes, I'll write a pair of SQL migration files and test them against a replica database.

When using migrations, always test the rollback migration. It is your emergency "oh shit" button.

John Millikin
+1  A: 

I use SQLylog to copy the structure, and I ALWAYS, let me repeat ALWAYS make a backup first.

Unkwntech
+1  A: 

I've used this tool before and it worked perfectly.

http://www.mysqldiff.org/

It takes as an input either a DB connection or a SQL file, and compares it to the same (either another DB connection or another SQL file). It can spit out the SQL to make the changes or make the changes for you.

+3  A: 

@[yukondude]

I'm using Perl myself, and I've gone down the route of Rails-style migrations semi-manually in the same way.

What I did was have a single table "version" with a single column "version", containing a single row of one number which is the current schema version. Then it was (quite) trivial to write a script to read that number, look in a certain directory and apply all the numbered migrations to get from there to here (and then updating the number).

In my dev/stage environment I frequently (via another script) pull the production data into the staging database, and run the migration script. If you do this before you go live you'll be pretty sure the migrations will work. Obviously you test extensively in your staging environment.

I tag up the new code and the required migrations under one version control tag. To deploy to stage or live you just update everything to this tag and run the migration script fairly quick. (You might want to have arranged a short downtime if it's really wacky schema changes.)

Lot105
+1  A: 

Pretty much what Lot105 described.

Each migration needs an apply and rollback script, and you have some kind of control script which checks which migration(s) need to be applied and applies them in the appropriate order.

Each developer then keeps their db in sync using this scheme, and when applied to production the relevant changes are applied. The rollback scripts can be kept to back out a change if that becomes necessary.

Some changes can't be done with a simple ALTER script such as a tool like sqldiff would produce; some changes don't require a schema change but a programmatic change to existing data. So you can't really generalise, which is why you need a human-edited script.

MarkR
+2  A: 

There are data comparison tools such as Toad or Data Compare tool that fasciliate pushing new functionality into different MySQL servers... are you aware of those?

Itamar
A: 

I've always preferred to keep my development site pointing to the same DB as the live site. This may sound risky at first but in reality it solves many problems. If you have two sites on the same server pointing to the same DB, you get a real time and accurate view of what your users will see when it goes live.

You will only ever have 1 database and so long as you make it a policy to never delete a column from a table, you know your new code will match up with the database you are using.

There is also significantly less havoc when migrating. You only need to move over the PHP scripts and they are already tested using the same DB.

I also tend to create a symlink to any folder that is a target for user uploads. This means there is no confusion on which user files have been updated.

Another side affect is the option of porting over a small group of 'beta-testers' to use the site in everyday use. This can lead to a lot of feedback that you can implement before the public launch.

This may not work in all cases but I've started moving all my updates to this model. It's caused much smoother development and launches.

Paulo
How do you handle renaming of columns? A quick rename would break your production environment, right?
Kevin Pang
@Kevin Pang - that's true but I don't rename my columns too often. I've found the lesser demon is to leave the column names the way they are. It makes me think carefully about column names but I also find it's a rare occurrence. If a rename must happen, a find-replace in two systems doesn't take much longer than one system. It still causes less downtime and aggravation than a DB migration. I know it's unconventional, but it works very well for me:)
Paulo
+1  A: 

The solution I use is another addendum to yukondude.

  1. Create a schema directory under version control and then for each db change you make keep a .sql file with the SQL you want executed along with the sql query to update the db_schema table.
  2. Create a database table called "db_schema" with an integer column named version.
  3. In the schema directory create two shell scripts, "current" and "update". Executing current tells you which version of the db schema the database you're connected to is currently at. Running update executes each .sql file numbered greater than the version in the db_schema table sequentially until you're up to the greatest numbered file in your schema dir.

Files in the schema dir:

0-init.sql 
1-add-name-to-user.sql
2-add-bio.sql

What a typical file looks like, note the db_schema update at the end of every .sql file:

BEGIN;
-- comment about what this is doing
ALTER TABLE user ADD COLUMN bio text NULL;

UPDATE db_schema SET version = 2;
COMMIT;

The "current" script (for psql):

#!/bin/sh

VERSION=`psql -q -t <<EOF
\set ON_ERROR_STOP on
SELECT version FROM db_schema;
EOF
`

[ $? -eq 0 ] && {
    echo $VERSION
    exit 0
}

echo 0

the update script (also psql):

#!/bin/sh

CURRENT=`./current`
LATEST=`ls -vr *.sql |egrep -o "^[0-9]+" |head -n1`

echo current is $CURRENT
echo latest is $LATEST

[[ $CURRENT -gt $LATEST ]] && {
    echo That seems to be a problem.
    exit 1
}

[[ $CURRENT -eq $LATEST ]] && exit 0

#SCRIPT_SET="-q"
SCRIPT_SET=""

for (( I = $CURRENT + 1 ; I <= $LATEST ; I++ )); do
    SCRIPT=`ls $I-*.sql |head -n1`
    echo "Adding '$SCRIPT'"
    SCRIPT_SET="$SCRIPT_SET $SCRIPT"
done

echo "Applying updates..."
echo $SCRIPT_SET
for S in $SCRIPT_SET ; do
    psql -v ON_ERROR_STOP=TRUE -f $S || {
    echo FAIL
    exit 1
    }
done
echo OK

My 0-init.sql has the full initial schema structure along with the initial "UPDATE db_schema SET version = 0;". Shouldn't be too hard to modify these scripts for MySQL. In my case I also have

export PGDATABASE="dbname"
export PGUSER="mike"

in my .bashrc. And it prompts for password with each file that's being executed.

Mike Howsden
A: 

Kevin nails the issue really. Sure, you could write scripts to migrate back and forth, but you can't be toying with the live database when developing. Just a quick rename breaks the live application.

There was a posting in the postgres mailing list about a migration tool that allows you to both develop and run the live database without errors. It may be worth exploring.

http://chronicdb.com

Gary