Problem using same instance of indexSearcher for multiple requests

Question

Hi, Am using Lucene API in a .net web application. I want to use the same instance of Indexsearcher for all the requests.Hence am storing indexsearcher instance in http cache.

here is my code for the same:

if (HttpRuntime.Cache["IndexSearcher"] == null)
                {
                    searcher = new IndexSearcher(jobIndexFolderPath);
                    HttpRuntime.Cache["IndexSearcher"] = searcher;
                }
                else
                {
                    searcher = (IndexSearcher)HttpRuntime.Cache["IndexSearcher"];
                }

When I execute the statement below, I get a runtime error :"Object reference not set to an instance of an object."

Hits hits = searcher.Search(myQuery);

What am i missing here?

Thanks for reading!

Answer 1

Two things:

1. If you're using .Net 2.0 and have not applied SP1, this may help.
2. Look at the problem this person was having.

Both entries refer to objects in the cache being expired too soon - almost immediately in both cases. Things might also be complicated by the fact that objects in the cache are not thread safe.

If you have to have a single IndexSearcher, why not provide it to the web app as a service?

Answer 2

My quick answer is...

You don't really need to use the same index searcher object for all request, in fact i would recommend against it. You only need to make sure there is only one thread updating the index.

If you really want one, how about a static member variable in the application that is initialized once and used by all?

The long answer is... I will try and find my code and see exactly how I handled the problem

Answer 3

I also have a web application that uses the Lucene API to query (my web app does not writes on the index) and I create a new instance of the searcher for every request. It might not be very "performant" but I never had that kind of problem.

If you'd like, my web app is on Google Code so you can download the source code and take a look at what I did. Here's the url to the project http://code.google.com/p/goomez/

Answer 4

Hey Steve,

Try something like the following:

protected static IndexSearcher searcher = null;
...

if (searcher == null)
{
    searcher = new IndexSearcher(jobIndexFolderPath);
}

Answer 5

First of all it's not safe at all, it should be:

var searcher = (IndexSearcher)HttpRuntime.Cache["IndexSearcher"];
if(searcher == null)
{
     searcher = new IndexSearcher(jobIndexFolderPath);
     HttpRuntime.Cache["IndexSearcher"] = searcher;
}

In your code cache can expire between check and assignment

Answer 6

Steve, see best ways of using IndexSearcher. This is a bit dated, but the principle remains: Use a single instance of IndexSearcher, guard it using the proper thread safe code (which I do not know how to do in .Net), and invalidate it once the index is updated. I believe this is what Jesse has suggested, and I second this idea.

Answer 7

Hi,

Instead of caching indexSearcher, am now caching IndexReader. If IndexReader is already in cache, am making a check if it is up to date.Otherwise am opening it and passing that instance to indexSearcher constructor.

Does this logic/code make sense wrt optimized search query response incase multiple requests are hitting the web server for search?

Thanks for reading.

string key = MyConstants.CacheKey.IndexReader;

            indexReader = MyCacheManager.Get<IndexReader>(key);

            if (indexReader == null)//cache is empty.open indexreader
            {
                indexReader = IndexReader.Open(myIndexFolderPath);
                MyCacheManager.Add(key, indexReader);
                indexSearcher = new IndexSearcher(indexReader);
            }
            else//cache contains indexreader...check if it is up to date
            {
                indexSearcher = base.GetIndexSearcher(myIndexFolderPath, indexReader);
            }
            protected IndexSearcher GetIndexSearcher(string indexFolderPath, IndexReader indexReader)
        {
            IndexSearcher indexSearcher = null;
            if (!indexReader.IsCurrent())//index is not up to date
            {
                indexReader = IndexReader.Open(indexFolderPath);

                indexSearcher = new IndexSearcher(indexReader);
            }
            else
            {
                indexSearcher = new IndexSearcher(indexReader);
            }

            return indexSearcher;

        }