Lucene.Net and I/O Threading issue

Question

Hello all,

I have an indexing function named "Execute()" using IndexWriter to index my site's content. It works great if I simply called it from a web page, but failed when I have it as a delegate parameter into System.Threading.Thread. Strangely though, it always work on my local dev machine, it only fails when I uploads to a shared host.

This is the error message I got

"Lock obtain timed out: SimpleFSLock error...."

Below is the failed code (but only fails on a shared host)

Scheduler scheduler = new Scheduler();
System.Threading.Thread schedulerThread = new System.Threading.Thread(scheduler.Execute);

Below is the code that works (work both on my local machine and on shared host)

Scheduler scheduler = new Scheduler();
schedulre.Execute();

Now, some ppl said, it could be a bad left over from the previous debugging session, so before I instantiated the IndexWriter, I did

if (IndexReader.IsLocked(indexingFolder))
{

    log.Debug("it is locked");
    IndexReader.Unlock(FSDirectory.GetDirectory(indexingFolder));
}
else
{
    log.Debug("it is not locked");
}

and guess what? my log says, it is not locked.

So now I'm pretty sure it's caused by the System.Thread.Threading, but I just have no clue as to how to fix it.

Thanks

Answer 1

Probably the worst one to try and answer this since I haven't used lucene / shared hosting, but SimpleFSLock sounds like it's locking the lucene index file by using an explicit lock file on the file system (not quite the same as locking in threading). I'd say check to make sure you have configured the proper file paths and that file permissions are set correctly.

Otherwise, hopefully someone more familiar with Lucene.net can answer.

Answer 2

Check that on the shared host, the thread has the same permissions to the index folder as you do on the development machine/shared host.

Update: You can find what Principal the thread is running under by interrogating the thread's CurrentPrincipal property. Though this is a read-write property, you may not have the permissions to set this property in your shared-host environment.

You might find this post helpful.

Answer 3

I believe the problem is with a write lock file in the Lucene index directory. Go and list the directory's files. In Java Lucene, you would have seen a file named write.lock in the index directory, meaning that the index was not properly closed last time (maybe a process was abruptly stopped). In Lucene.net, look for a similarly named empty file. I believe the same mechanism will be used in Lucene.net. Try finding that file, erasing it and restarting Lucene.net.

Answer 4

Thanks everyone and especially to Vinay for pointing me in the right direction. After much tracing, i finally decided to take a look at the source and see what's there.

In "IndexWriter", you have

  Lock @lock = this.directory.MakeLock("write.lock");
  if ([email protected](this.writeLockTimeout))

which is pointed to the SimpleFSLock implementation. The culprit was

new FileStream(this.lockFile.FullName, FileMode.CreateNew).Close();

by creating a new thread, internally, it throws a system.unauthorizedaccessexception, according to msdn here

When starting a new thread, System.Security.Principal.WindowsIdentity.GetCurrent() returns the identity of the process, not necessarily the identity of the code that called Thread.Start(). This is important to remember when starting asynchronous delegates or threads in an impersonated ASP.NET thread.

If you are in ASP.NET and want the new thread to start with the impersonated WindowsIdentity, pass the WindowsIdentity to the ThreadStart method. Once in the ThreadStart method, call WindowsIdentity.Impersonate().

Such, I solved my issue by impersonate the IIS account running my application in "Execute()" function and all problems are resolved.

Thanks again to all.