Lucene.Net maintaining indexes when using MultiSearcher

Question

In order to keep my index up to date, I need to add / modify my search index every 5 minutes. The way I have it set up is with 2 indexes, one in the Full directory and one in the Incremental directory, and to search them I am using a MultiSearcher.

I am now writing a process to maintain the index. What I am doing is passing the last index date to a stored procedure and the DB is returning all new / modified records based on an "UpdatedOn" field in the DB. I then loop through the 2 directories, opening an IndexReader for each directory and deleting the document based on a TermQuery for the primary key. I then add the records to the incremental index using an IndexWriter and optimize it.

What's happening when I do subsequent searches though is that records that I know are in the index are not being returned.

Am I doing the index maintenance wrong?

Answer 1

Is your {Multi}IndexSearcher being closed and re-opened on the search interface? For performance reasons, I've seen some applications persist the SearchIndexer object in memory to avoid the overhead of re-opening the object and its underlying directory with every search, however, in doing so, the IndexSearcher doesn't pick up changes/additions to the underlying index.

I'm not sure if this is what is happing in your case - but usually when I've seen a searcher not picking up new documents, it is because the searcher was opened before the item was added to the index ... just a thought.