Synchronizing Lucene indexes across 2 application servers

Question

I've an asp.net web application hosted on a web server(IIS 7).It uses Lucene for search functionality. Lucene search requests are served by .Net WCF services sitting on 2 application servers (IIS 7).The 2 application servers are Load balanced using "netscaler".

Both these servers host a .net windows service which updates search indexes on the respective servers in the night on a daily basis.

I need to synchronize search indexes on these 2 servers so that at any point of time both the servers have uptodate indexes. I was thinking what could be the best architecture/design strategy to do so given the fact that any of the 2 application servers could be serving search request depending upon its availability.

Any inputs please?

Thanks for reading!

Answer 1

Basically you need two identical copies of the same Lucene index - one for each IIS server. I believe the simplest approach is to build an updated index on one machine, optimize it and then copy it to the other machine. On Linux I would use rsync, but I do not know the Windows equivalents. See Jeff Atwood's ideas for Windows rsync alternatives. Alternatively, you could issue the same index update commands to both Lucene indexes and verify they were processed properly. This is harder technically and only useful when you have more frequent updates. Please see Scaling Lucene and Solr for a broader discussion of distributed Lucene indexes.