-- I don't want to start any religious wars, but a quick google search indicates that Apache Lucene is the preferred open source tool for indexing and searching. Are there others?
-- What file format does Lucene use to store its index file(s)?
Thank is advance. Doug
http://stackoverflow.com/questions/602787/which-are-the-best-alternatives-to-lucene And as a lucene user I can say it has improved a lot performance wise the last couple of versions (NOT meaning it was slow before!)
it uses an proprietary format see here
I suggest you to look at Sphinx. I have an experience with Lucene.net and we have many problems with multithread indexing. Lucene stores index in files, and this files can be locked by anti-viruses software. Also you can not compare numbers in Lucene: it is impossible to filter products by size and price.