SharePoint Development and Data Centric Projects

Question

I was involved in a SharePoint(WSS) project that was very data centric. The project consisted of more than 500 lists that has very complex relations between them. The client also asked for more than 350 Reports. Don't tell me why did you use SharePoint from the beginning. It was a managerial decision and we already delivered the project after 14 months of pain (this is 6 months overdue to the deadline)

When we first started the project, we didn't know anything about SharePoint development (believe it or not). The management said that they will take the risk. They were very convinced that SharePoint is the optimal solution for anything!!!(well, that proved wrong at the end of the project).

Anyway, we were learning SharePoint while we were developing. Our development were mainly based on SharePoint designer to customize all the AllItems/NewForm/EditForm/DispForm for every list to provide the needed logic/validation that the client asked for (Using JavaScript). We also implemented around 15 Custom Fields (e.g. master-details fields). We also made an event receiver to handle all the adding/updating/deleting... events for all the lists in the site. Plus around 40 ASP.Net user controls.

The main problems that faced us (we worked-around it but sadly in an inefficient way)

1- The Client asked for a search web part in each AllItems.aspx. The search web part should have multiple keys for the client to search with. We did that using Form Web parts using SPD no problem. But the real problem was how to search for a related field that was not in the current list. (So, in such cases we had to save these fields values in our list to be able to search for (crap, I know!!)). You might ask, why didn't you implement ASP.Net user controls for such task? Well, that would require us to forsake the default AllItems web part, and were already customized hundereds of AllItems.aspx pages with alot of customization that would take us a lot of time to reimplement them from the beginnging. Also, even if we used user controls, CAML is very inefficient in retrieving data from multiple related lists!

2- I think you can guess this one, if we've already faced a big time doing search web parts, how on earth will be able to do the 350 reports!!:D But we figured out a work-around (as usual :S) we made an Access DB file with links to all the 500 sharePoint lists, then we implemented a user control that has a report viewer control. This user controls takes an ordinary T-SQL Query to query on the Access DB, the Access DB retrieves the data from the SharePoint DB and pass it back to the user control which views the DataSet on the report viewer.

There are other administration related problems, but I would like to focus on the development here.

So, after I showed you the picture (sorry for the long post). What do you think was the best SharePoint development technique that we should have taken in such a data centric Project, if any?

I heard that some companies doesn't use lists at all in such projects, and builds there own SQL database tables instead of the SharePoint Database. But I can't keep my self from wondering, If I'm making my own DB, and hence implementing my CRUD web parts from scratch (We will also lose the security module benefits provided by SP Lists), what would be the benefits of SharePoint?

Once again I apologize for the long post.

Answer 1

I think you found out exactly what I did. Sharepoint just isn't good at handling large enterprise type applications. We ended up creating a custom database to house our data. We used Webparts for the user interface, but otherwise, the entire application was independent of Sharepoint.

In my opinion, Microsoft is overselling Sharepoint. It's actually good at team collaboration sites and simple Excel services applications, but anything beyond that it just isn't capable of handling.

Answer 2

I disagree with Geoff that SharePoint is no good for large enterprise type apps. You have to remember from the start that SharePoint is a development PLATFORM. This means it gives tyou a lot of functionality out of the box, but is extremely customisable.

Being a platform does not mean every bit of customisation needs to be done based on SharePoint lists. Seeing as it is built on ASP.NET you can do anything in SharePoint you could do in ASP.NET as well.

I have built a great deal of ASP.NET apps that were hosted in SharePoint, letting SharePoint do the authentication etc.

I have to say though, determining where SharePoint should stop being your base and you should switch to regular ASP.NET is sometimes hard..

Answer 3

If you are looking for a data-centric solution in SharePoint, the best solution is to use the Business Data Catalog (BDC). This keeps your rich data relationships and all of the SQL (or other DBMS) goodness you want where it should be - in a repository designed to be optimal at storing data.

For an overview of what the BDC functionality can do, see this post on the SharePoint Team Blog. For much more details, read the series on SharePoint Magazine. Note that these features requires an Enterprise license of SharePoint 2007.

Answer 4

I have to disagree with both Geoff and Abu in regards to SharePoint being a bad choice for large enterprise applications.

As you state yourself Abu your team were learning on the job as you had no SharePoint development experience, the issues you faced was more a Management error that a platform problem, management should have brought in SharePoint contractors to work alongside your team to help build what sounds to be a fairly complex system.

As a developer who has worked with SharePoint for a number of years many of the projects I have worked on some that I myself would not have believed suitable for SharePoint with in my first few years of developing on this platform, however now with more experience I know how to leverage the power of the platform far better and I realise the advantages gained using SharePoint for projects of that nature. That said I have a number of issues with parts of the platform but this is no different to any other platform I have worked on including parts of the ASP.Net platform.

If I was asked to develop a solution using a bespoke Java based system (or perhaps the new MVC platform) I am sure I would experience many problems similar to what you experienced where I simply don’t know what the right approach would be. That would not in any way be an issue with the platform but more with my inexperience.

I am sorry to hear that both of you experienced pains working within the bounds of the SharePoint platform that was forced on you by your management. Though I am disappointed that you are so fast to point the blame away from yourselves and your management.

Was SharePoint the best platform for your projects I can’t say, but that doesn’t make it a bad platform for enterprise applications.