views:

24

answers:

1

In SharePoint 2010, we have the List View Threshold, which defaults to 5,000, and:

Specifies the maximum number of list or library items that a database operation, such as a query, can process at the same time outside the daily time window set by the administrator during which queries are unrestricted.

So, I'm looking to determine the options for querying a document library that will have more items than the threshold limit.

Ignoring the 'unrestricted' dailty time window, will the threshold be triggered under the following scenarios on a document library with more than 5,000 items?

  1. Using CAML Queries in Content Query WebParts on indexed columns, non-paged results (Answer: Yes)
  2. Using CAML Queries in Content Query WebParts on indexed columns, paged results (100 per page)
  3. Using the Search API on indexed or non-index columns (Answer: No)
A: 

I have not found good documentation which has "finite list" of operations that triggers the threshold limit. But this blog post is good: http://blogs.technet.com/b/speschka/archive/2009/10/27/working-with-large-lists-in-sharepoint-2010-list-throttling.aspx

But I would consider increasing threshold limits via Central Administration because if your have list/document library exceeding the threshold, unusual things starts occuring. For example, you may be blocked from doing this operation because accessing that many items could adversely affect other users of the site.

As far as No 3. is concerned - I would say "NO" because the search API does not query the list directly but it looks into its own index and there is no practical limit on that!

Another very useful resource: http://office.microsoft.com/en-us/sharepoint-foundation-help/manage-lists-and-libraries-with-many-items-HA010377496.aspx#_Toc264017710

It is always advised to use ContentIterator which provides methods to iterate list items, lists, sites to regulate the amount of data that is transferred (i.e., to avoid throwing a SPQueryThrottledException). Also, this training video will help you a lot in answering some of your questions: http://msdn.microsoft.com/en-us/sharepoint/ff420380.aspx#lesson2

Ashish Patel
Ashish, increasing threshold limits only avoids the issue which will have to be dealt with at a later date; I do not wish to circumvent the throttling, only understand how to retrieve data when it is in effect.
Jon Schoning
I agree with you. I think for your no 2, the behavior should be same (answer=yes) because CAML query is used by views. Apart from that I am sure you would have seen "Working with Large Folders and Lists" topic in SDK. Also you can take an approach of returning only a couple of thousand records to your users and suggest them to refine your search criteria if the things they are looking for is not found in those few thousand records.
Ashish Patel
Great resources, thanks. Does No 2 = Yes even if it is paged so that it only displays 100 items on each page displayed to the user?
Jon Schoning
Let me correct this a little bit. The correct answer is "it depends". You can have a million items in List/Library with default 5000 threshold limits and it would work fine as long as you access the items correctly. the resources link in my answer would help to understand how should you write queries or access the list. To be specific, if you use ContentIterator object for paging, you are okay.
Ashish Patel