How to find related items by tags in Lucene.NET

Question

My indexed documents have a field containing a pipe-delimited set of ids:

a845497737704e8ab439dd410e7f1328|
0a2d7192f75148cca89b6df58fcf2e54|
204fce58c936434598f7bd7eccf11771

(ignore line breaks)

This field represents a list of tags. The list may contain 0 to n tag Ids.

When users of my site view a particular document, I want to display a list of related documents. This list of related document must be determined by tags:

• Only documents with at least one matching tag should appear in the "related documents" list.
• Document with the most matching tags should appear at the top of the "related documents" list.

I was thinking of using a WildcardQuery for this but queries starting with '*' are not allowed.

Any suggestions?

Answer 1

Your pipe-delimited set of ids should really have been separated into individual fields when the documents were indexed. This way, you could simply do a query for the desired tag, sorting by relevance descending.

Answer 2

You can have the same field multiple times in a document. In this case, you would add multiple "tag" fields at index time by splitting on |. Then, when you search, you just have to search on the "tag" field.

Answer 3

Setting aside for a minute the possible uses of Lucene for this task (which I am not overly familiar with) - consider checking out the LinkDatabase.

Sitecore will, behind the scenes, track all your references to and from items. And since your multiple tags are indeed (I assume) selected from a meta hierarchy of tags represented as Sitecore Items somewhere - the LinkDatabase would be able to tell you all items referencing it.

In some sort of pseudo code mockup, this would then become

for each ID in tags
  get all documents referencing this tag
  for each document found
    if master-list contains document; increase usage-count
    else; add document to master list
sort master-list by usage-count descending

Forgive me that I am not more precise, but am unavailable to mock up a fully working example right at this stage.

You can find an article about the LinkDatabase here http://larsnielsen.blogspirit.com/tag/XSLT. Be aware that if you're tagging documents using a TreeListEx field, there is a known flaw in earlier versions of Sitecore. Documented here: http://www.cassidy.dk/blog/sitecore/2008/12/treelistex-not-registering-links-in.html

Answer 4

Try this query on the tag field.

+(tag1 OR tag2 OR ... tagN)

where tag1, .. tagN are the tags of a document.

This query will return documents with at least one tag match. The scoring automatically will take care to bring up the documents with highest number of matches as the final score is sum of individual scores.

Also, you need to realizes that if you want to find documents similar to tags of Doc1, you will find Doc1 coming at the top of the search results. So, handle this case accordingly.