A few Lucene questions

Question

I've been using Zend and need a search. The Zend docs aren't great so I had a couple questions that are easy to answer but not directly obvious. I'm using Lucene to search an SQL database

1. How do I associate the index of my item with the text of that item. So if they search and find the item, how do I get its index returned? As far as I can tell you can only return the text of the search.

2. When I add an item to the document that holds all the data, but the document is created already, is it simply a open('document'), $doc = new Doc(), $doc->addDocument(), commit()?

3. I understand that I update the Lucene document every time that I add something to the database. In optimizing, should I reoptimize every time that I add something? Is that inefficient? Should I do it once a week?

Sorry to ask what seems like obvious questions, and thanks for your help in advance.

Answer 1

1. 'Index, and thou shalt retrieve' - You have to index what you want to be returned eventually. That is, if you want to be able to return record id 1389 when searching for its text "Flux Capacitator", you should store a document having the text in one field and the id in another field. The id field does not have to be indexed, but it has to be stored so you can get it back.
2. What you are looking for is an 'update document' action. Lucene does not really have them. You should delete the document first, and then add a new document containing the updated information. Now go back to item 1, take the id field you added there and make it indexed (say as Keyword), because you will need to use it as a unique identifier of the document in order to delete it.
3. Great question. This is very much dependent on your use case. Do you have a daily "dead time", when your site/database is relatively idle? That would be the time to optimize. Do you have no such time? You can forgo optimizing and take a small (say 5-10%) performance penalty, which could also be mitigated using the Merge Factor.

I hope this make sense. If it does not, please ask in the comments.

Answer 2

point 3) is addressed in Lucene 2.9 as NRT(NearRealtimeSearch) implemented by means of SegmentReader + internal RamDirectory usage

check OtisGospodnetic wiki entry