Script SharePoint crawl rules

Question

I want to exclude certain pages from MOSS indexing like a confirmation page that sits in the pages library in the root of my site: http://server/Pages/ConfirmSignup.aspx

I can do this by going to search administration / search result removal and adding the url to the URLs to remove box.

Because I have dev, staging, uat, production environments I want to script this. I could only find a command in Gary Lapointe's stsadm commands but that adds an exclusion to a search scope which does not seem to work for individual files, only folders. Since there are other files in my /Pages library I can't use this.

How do I add search result removal urls programmatically?

Answer 1

The SPList object has a NoCrawl property. Setting this to true will ensure no items in the list will be indexed or appear in search results.

Unfortunately this doesn't go down to the SPListItem level. You would need to have an 'Admin' site and exclude its Pages list from indexing.

The advantage this solution has is its level of control. In some cases crawl rules are very complex or impossible to define correctly in search configuration. This option avoids those issues.