Zend_Search_Lucene, how to share an index storage folder over network

Question

Hi,

I am running web application on two different servers with load balancing, and using Zend_Search_Lucene for indexing documents.

Now I am facing indexing issue which a user comes to the site through server #1 and stores information, Zend_Search_Lucene stores index only server #1.

So once another user comes to the site through server #2 and search, only the indexes in server #2 retrieve.

Is there any way to share an index storage folder to be accessed by both servers? Or is there any way to store indexes in mysql database, not a file system?

Answer 1

okay please don't hurt me because i won't give you an exact answer to your question but give you an alternative.

please dont use zend for that, its very slow. believe me. its just a bad implementation that has to be fully loaded for every search request. just do some benchmarks and you'll notice.

lucene does indeed support sharding (distributing indexes to several servers and combine the results seperately fetched)

it does as well support replication which would be a better go for you

so here comes the actually useful part:

check out solr at http://lucene.apache.org/solr/ its all setup witn an example installation using jetty included. you raelly don't have to do anything more than customize your fields if you dont use wildcard fields and start it using java -jar start.jar

once the server is running on its default port which i just forgot, you have an inderface to which you can send your new documents using http post. you can do that from php from each of your application servers. solr will then index it and commit the changes if you included the commit flag.

you can query your index using the http interface and the solr query parameters. then you get a neat json or xml formattet document. you can replicate your server if you experience any load problems.

but your server will manage 1000x times more requests than if you do it using zend because everything is up and running, lucene is loaded and the querys can be prozessed right away.

just check out this step by step tutorial and you should be fine: http://lucene.apache.org/solr/tutorial.html

if you dont like all that and want to stick with zend anyway:

database is not possible bei nature, but you could write a wrapper. this would however be absurd. the best thing to do in this case is to set up a network file system!

any questions, lemme know.

ps: when you come across the choice, is suggest using dismax, in most cases its a lot faster.