Is there a Ruby database migration gem, that helps you move content from an old structure to a new structure?

Question

Hi there,

Are there any Ruby gems/libraries that help you migrate from an old DB structure to a new structure? ActiveRecord migrations do a good job keeping track of a new db structure, but I wonder if there's something that helps you migrate an entire legacy db to a new structure:

transfer_from(:source_table => 'person', :destination_table => 'dudes_and_dudets') do

  from :name, :to => :full_name

  from :dob, :to => :age do |dob|    # this would, for example, load the result  
    (Date.today - dob)/60/60/24/365  # of the block into :age
  end

end

(I realize you could do these transforms just as easily with AR, but I'm hoping the magic library would have many more transforms.

Berns

Answer 1

1. config your database in config/database.yml
2. rake db:schema:dump
3. convert schema.rb into db/migrate/001_create_database.rb
4. you can generate other migration to maintain your database schema

Answer 2

There are quite a few for moving databases without transforms. I remember the Rails Envy guys talking about a gem (but it's a while back and I haven't got time to go digging). Have a look on railsenvy.com?

Answer 3

You can access all your models from within a migration, and thus handle all your data migrations right there too. If you already knew this, and your question was about a neater way of doing it, then of course this is not the answer you're looking for.

One problem with your example is that you can't migrate down to an earlier version, but only because of the block feature you demonstrate in conversions.

I admit that your example is nice and terse, but here's a regular migration example any way:

class FooBar < ActiveRecord::Migration
  def self.up
    # This is only needed if the new table will have the same name.
    # Move the old one aside.
    rename_table :users, :old_users

    # The new table structure
    create_table :users do |t|
      t.string :full_name
      t.date   :age
    end

    # Data migration
    OldUsers.all.each do |orig|
      User.create!(
        :full_name => orig.name,
        :age => (Date.today - orig.dob)/60/60/24/365
      )
    end

    # Clean up
    drop_table :old_users
  end

  def self.down
    # Exercise for the reader!
  end
end

# Temporary class for accessing the old table during conversion
class OldUsers < ActiveRecord::Base; end

Answer 4

I've had to do something like (what I think) you're describing, and I used Sequel for it. Sequel is adaptable and generally useful and can work with SQL directly in a fairly handy fashion and can access multiple different sorts of DBs.

The documentation is very handy, and I recommend it entirely.

Here's an example of using sequel to take a huge arbitrarily flatfile from geonames and use it to fill out a db and make queries. This is probably a good example for you to make something to do what you want.

It's rails-agnostic. doesn't need to be attached to a model, migration, or anything else other than a couple gems.

Answer 5

I've begun working on this.

If anyone would like to give tips on a better/more idiomatic, or more efficient implementation, please let me know.

http://github.com/btelles/legacy_migrations

edit:

I now have this exact syntax working on the above github repository... plan to add a few rake tasks for mapping the old structure to new ActiveRecord classes, and more transforms...in case anyone's interested.

It's also on gemcutter/rubygems: gem install legacy_migrations