How to partially migrate a database to a new system over time?

Question

We are in the process of a multi-year project where we're building a new system and a new database to eventually replace the old system and database. The users are using the new and old systems as we're changing them.

The problem we keep running into is when an object in one system is dependent on an object in the other system. We've been using views, but have run into a limitation with one of the technologies (Entity Framework) and are considering other options.

The other option we're looking at right now is replication. My boss isn't excited about the extra maintenance that would cause. So, what other options are there for getting dependent data into the database that needs it?

Update:

The technologies we're using are SQL Server 2008 and Entity Framework. Both databases are within the same sql server instance so linked servers shouldn't be necessary.

The limitation we're facing with Entity Framework is we can't seem to create the relationships between the table-based-entities and the view-based-entities. No relationship can exist in the database between a view and a table, as far as I know, so the edmx diagram can't infer it. And I cannot seem to create the relationship manually without getting errors. It thinks all columns in the view are keys.

If I leave it that way I get an error like this for each column in the view:

Association End key property [...] is not mapped.

If I try to change the "Entity Key" property to false on the columns that are not the key I get this error:

All the key properties of the EntitySet [...] must be mapped to all the key properties [...] of table viewName.

According to this forum post it sounds like a limitation of the Entity Framework.

Update #2

I should also mention the main limitation of the Entity Framework is that it only supports one database at a time. So we need the old data to appear to be in the new database for the Entity Framework to see it. We only need read access of the old system data in the new system.

Answer 1

You can use linked server queries to leave the data where it is, but connect to it from the other db.

Depending on how up-to-date the data in each db needs to be & if one data source can remain read-only you can:

1. Use the Database Copy Wizard to create an SSIS package that you can run periodically as a SQL Agent Task
2. Use snapshot replication
3. Create a custom BCP in/out process to get the data to the other db
4. Use transactional replication, which can be near-realtime.

If data needs to be read-write in both database then you can use:

1. transactional replication with update subscriptions
2. merge replication

As you go down the list the amount of work involved in maintaining the solution increases. Using linked server queries will work best if its the right fit for what you're trying to achieve.

EDIT: If they're the same server then as suggested by another user you should be able to access the table with servername.databasename.schema.tablename Looks like it's an entity-framework issues & not a db issue.

Answer 2

I don't know about EntityToSql but I know in LinqToSql you can connect to multiple databases/servers in one .dbml if you prefix the tables with:

ServerName.DatabaseName.SchemaName.TableName 

MyServer.MyOldDatabase.dbo.Customers

I have been able to click on a table in the .dbml and copy and paste it into the .dbml of the alternate project prefix the name and set up the relationships and it works... like I said this was in LinqToSql, though have not tried it with EntityToSql. I would give it shot before you go though all the work of replication and such.

Answer 3

If Linq-to-Entities cannot cross DB's then Replication or something that emulates it is the only thing that will work.

For performance purposes you probably want either Merge replication or Transactional with queued (not immediate) updating.

Answer 4

Thanks for the responses. We're going to try adding triggers to the old database tables to insert/update/delete records in the new tables of the new database. This way we can continue to use Entity Framework and also do any data transformations we need.

Once the UI functions move over to the new system for a particular feature, we'll remove the table from the old database and add a view to the old database with the same name that points to the new database table for backwards compatibility.

One thing that I realized needs to happen before we can do this is we have to search all our code and sql for @@Identity and replace it with scope_identity() so the triggers don't mess up the Ids in the old system.