Save Sharepoint Data outside of Lists

Question

What is the best way to store your Data outside of SharePoint. I want to use the default Edit/View options in SharePoint (though I do plan on extending their functionality). I need to store the data outside of the SharePoint Lists as I am expecting a large record set(150,000 to start with).

Answer 1

I totally agree with GalacticJello, storing data outside of the regular sharepoint content database is at the moment (MOSS 2007) a complete and utter nightmare. MS provided a ExternalStorage provider baseclas for us to override, but there are major cons against using this:

• writing and using your own custom ExternalStorage provider implementation is doable, but very difficult
• ExternalStorage provider implementations only do just that, making sharepoint store stuff in a different location than the regular content db.

• This means that you would need to write your own code to keep the external storage and the list items in sync, and i'm not even mentioning workflows and versioning.

• last but not least (and IMHO the worst), creating and using a custom ExternalStorage provider is not web app or site collection targetable, it's usage is farm wide (and there are NO workarounds). So any site collection you create in that farm will have it's doc libs use the external storage provider.

Answer 2

You can store the items in the list, the trick is to create efficient views that return paged data quickly back to the user.

Another option is to use folders to split up the data.

If you really need to store it outside of SharePoint, I would consider waiting for SharePoint 2010 and thier "External Lists" features, as there are a ton of pitfalls and things to consider if you want to mimic that functionality in SharePoint 2007 (been there, done that).

Answer 3

You could create a SQL Data source and use the Data Form Web part to connect it to an edit form etc.:

Data Forms and SQL Server, Part 1

Data Forms and SQL Server, Part 2

Personally I prefer to have the level of control of a totally custom web part.. with this you will not run into limitations eventually.

Answer 4

In this case it sounds like all the elements will be in the same list, otherwise SLAM (free tool on CodePlex) is a very neat data replication tool for related lists.

Based on the comments above it sure looks like a plain ol' ASP.NET page might be the best option, unless there is some good reason why SharePoint must be used.