Best practices for implementing a Lucene search in asp.net eCommerce site

Question

I've been tasked with seeting up a search service on an eCommerce site. Currently, it uses full text indexing on sql server, which isn't ideal, as it's slow, and not all that flexible.

How would you suggest i approach changing this over to lucene? By that, i mean, how would i initially load all the data into the indexes, and how would it be maintained? on my "insert product" methods, would i also have it insert it into the index?

any information is of great help!

Answer 1

I'm currently using Solr, which is build on top of Lucene, as the search engine for one of my e-commerce projects. It works great.

http://lucene.apache.org/solr/

Also as far as keeping the products in sync between the DB and Solr, you can either build your own "sweeper" or implement the DataImportHandler in Solr.

http://wiki.apache.org/solr/DataImportHandler

We build our own sweeper that reads a DB view at some interval and checks to see if there are new products or any product data has been updated. It's a brute force method and I wish I knew about the DataImportHandler before.

Facets are also a really powerful part of Solr. I highly recommend using them.

Answer 2

If you do decide to use Lucene.NET for your search you need to do some of the following:

• create your initial index by iterating through all your records and writing the data that you want searched into your index
• if the amount of records and data that you are writing to your indexes makes for large indexes then consider stuffing them into multiple indexes (this means you will have to make a more complex search program as you need to search each index and then merge the results!!)
• when a product is updated or created you need to update your index (there is a process here to create additional index parts and then merge the indexes)
• if you have a high traffic site and there is the possibility of multiple searches occurring at the exact same moment then you need to create a wrapper that can do the search for you across multiple duplicate indexs (or sets of indexes) (think singleton pattern here) as the index can only be accessed (opened) for one search at a time

This is a great platform. We initially tried to use the freetext search and found it to be a pain to create the indexes, update, and manage. The searches were not that much faster than a standard sql search. They did provide some flexibility in the search query...but even this pales in comparison to the power of Lucene!