Sharepoint: Integrity of lookup fields after a list import

Question

Hi there

I got a question about the behavior of lookup fields when importing data. I wonder how the lookup fields behave when the list they point to is being replaced/imported. To explain the issue, I will provide a quick example below:

As example, assume we have these two sharepoint lists:

Product Types
-------------
+ Type Name
+ Code Nr
+ etc


Products
--------
+ Product Name
+ Product Type (Lookup field to list "Product Types")
+ etc

In my scenario, the Products List contains production data on the production Sharepoint platform. It is filled with data by the business users. However the Product Types list contains rather static data and is maintained by the developer.

Now after a development cycle, the developer wants to deploy his new webparts and his new data (product types list). The developer performs the following procedure:

1. On the dev machine: Export "product type" list using stsadm
2. On the production machine: Delete all items in the "product type" list
3. On the production machine: Import the "product type" list using stsadm

This means we basically replace the "product type" list on the production server while keeping the "product" list as it is.

Now the question:

• Is this safe? Will the lookup references break under certain circumstances?
• Any downside of this import/export procedure?
• What happens if someone accesses a "product" during the import? Will the (now invalid) reference clear its own content (become a null value).
• What happens if the schema of the "product type" list changes (new column)? Will this cause any troubles?

Thanks for all feedback and suggestions!

Update 1 The imported "product type" items have the same IDs as previously deleted ones.

Update 2 Started a bounty to get some more feedback/opinions.

Answer 1

There are two components to a particular lookup: the field, and the field value. The field value only contains the ID of the item(s) it refers to, and the display field. This information is meaningless without the field, which specifies what list to look at and what field to use as the display field.

• The primary reason that a Lookup will break occurs on the field scope: either the list it referred to no longer exists, or the list does not contain the required field. These would generally happen if you deleted and recreated the list, but you aren't doing that. If you do break a lookup's list reference, then the only thing you can do is re-create the lookup, because you cannot configure the list reference for a lookup field once it is created.
• The downside of your import/export procedure is that you lose the validity of all currently existing lookup values. A lookup maintains its integrity based on the ID of the item it references. So when the display field changes, it still refers to the same item. If you delete the item, then the lookup no longer references it, even if you create a new item that has the same value for the display field. So you would have to reassign all of the products to the new product types.
  • It should be noted that if you were to revert the deletion of that item, it would return to being on the lookup! The reference to that ID is kept until the actual lookup value is updated (such as by editing the Product).
• All of your now invalid references will be null for purposes of interaction. You won't see anything on display forms, and you won't have the options when you try to update the product. When you do update the product, you update it to what you just set it to, which since you can't set the non-existent IDs, means that there are no more references to those IDs.
• Any changes to the Product Type list's schema that do not affect the display field specified for the lookup will not have any effect on the lookup integrity. If you do change the display field in any fashion, and of course if you delete it, then it will break in the same fashion as with the list reference. However, you can set the display field, both in the UI and in the object model, so it is easy to fix this.

Answer 2

We have had this exact same scenario before. This is a little tricky, depending upon how you will approach it.

1) Delete and Recreate Product Type list through UI

If you delete and recreate the lookup List(Product Type in your case) through UI, then you will lose the connections because the List's id GUID will change upon recreation. So do not go that route.

2) Delete and Recreate Product Type through a Feature

If you had created the Product Type list through a feature.xml file using the <ListInstance> element, then if you delete that list and then recreate it using the same feature (basically Id attribute of ListInstance remains the same, number of list items, i.e. the number of <Row> elements, may change), the association would be maintained. So if you were adding 5 more product types, then if you had created the list using a feature, you could just delete the list and provision the new one using the same feature with extra info for new items and everything would just work!

As a side note, this is the better approach because if you have to do the upgrade on a lot of servers, then rather than doing list export import via stsadm, feature deactivation and activation is a much more recommended solution. This is how we did it.

3)Deleting all list items from Product Type and adding new ones (list is never deleted)

If you are linking the lookup field (in Product List) to the ID field of the lookup list(Product Type), you have to remember that ID is auto-incrementing, so if you delete all items and then add new ones, then their ID's would be different. Say you had 5 items with ID's (ID field is not shown in UI while editing in Datasheet view) 1-5 in the list. If you delete them and add new items, their ID's would start from 6 and not 1 again. So if your lookup field had link to the item with ID 1 in it, then this method is not going to work because there is no item with ID 1 in the Product Type list anymore. So you might want to really try this out before going to production with this method.

4) Editing the list in place

If the list is not extraordinarily huge, and you only have to make this change to one or two instances, could you not just edit the list directly in the datasheet view on the prod server? When editing in datasheet view, do not delete the item, but just overwrite the values of its columns. And you can add more items if you want. This will make sure your ID's are valid.

I have mostly talked about adding new items to the list. Now if you were deleting existing items, then your lookup fields will be affected because assuming you linked the field by ID, the ID is not present anymore since the item has been deleted. Basically, any method you use, maintaining your ID's is critical.

Now regarding your doubts/questions: I am not too sure about stsadm export import for a list (never done it myself), but stsadm can be tricky as some operations will work on certain scopes only. So you better try out your exact scenario on a dev env.

What happens during an import is tricky again depending on the exact timing. I am sure SP has its own concurrency mechanisms, but you cannot have a definitive answer as it might probably be different based on the stage of the import. If possible, recommended approach is to do the import during a planned downtime.

Regarding changing schema of the list, a change in the schema of a list will not affect the existing list instances (for the most part). If you do this through UI, I believe SP makes changes to the content DB directly. I am not certain how you intend to do this, but if you were to add a column to an existing list using a feature, the way to do this is during feature activation by adding a new content type to the list and adding your new column to this content type. This way you add the column but do not affect the existing list items.

Good luck...