search integration

Question

Hi there, I am working on a website that currently has a number of disparate search functions, for example:

• A crawl 'through the front door' of the website
• A search that communicates with a web-service
• etc...

What would be the best way to tie these together, and provide what appears to be a unified search function?

I found the following list on wikipedia

Free and open source enterprise search software

• Lucene and Solr
• Xapian

Vendors of proprietary enterprise search software

• AskMeNow
• Autonomy Corporation
• Concept Searching Limited
• Coveo
• Dieselpoint, Inc.
• dtSearch Corp.
• Endeca Technologies Inc.
• Exalead
• Expert System S.p.A.
• Funnelback
• Google Search Appliance
• IBM
• ISYS Search Software
• Microsoft (includes Microsoft Search Server, Fast Search & Transfer):
• Open Text Corporation
• Oracle Corporation
• Queplix Universal Search Appliance
• SAP
• TeraText
• Vivísimo
• X1 Technologies, Inc.
• ZyLAB Technologies

Thanks for any advice regarding this.

Answer 1

Take a look at these similar questions:

• Best text search engine for integrating with custom web app?
• How do I implement Search Functionality in a website?

My personal recommendation: Solr.

Answer 2

All these companies offer different features of Universal Search. Smaller companies carved themselves very functional and extremely desired niches. For example Queplix enables any search engine to work with structured data and enterprise applications by extracting the data, business objects, roles and permissions from all indexed applications. It provides enterprise-ranking criteria as well as data-compliance alerts.

Answer 3

Solr is an unbelievably flexible solution for search. Just in the last year I coded 2 solr-based websites and worked on a third existing one, each worked in a very different way.

Solr simply eats XML requests to add something to index, and XML requests to search for something inside an index. It doesn't do crawling or text extraction for you, but most of the time these are easy to do. There are many existing addons to Solr/Lucene stack so maybe something for you already exists.

I would avoid proprietary software unless you're sure Solr is insufficient. It's one of the nicest programs I've worked with, very flexible when you need it and at the same time you can start in minutes without reading long manuals.

Answer 4

Note that no matter what search solution you use, a search setup is "disparate" by nature. You will still have an indexer, and a search UI, or the "framework".

You WILL corner yourself by marrying a specific search technology. You actually want to have the UI as separate from the search backend as possible. The backend may stop to scale, or there may be a better search engine out there tomorrow.

Switching search engines is very common, so never - ever - write your interface with a specific search engine in mind. Always abstract it, so the UI is not aware of the actual search technology used.

Keep it modular, and you will thank yourself later.

By using a standard web services interface, you can also allow 3rd parties to build stuff for you, and they won't have to "learn" whatever search engine you use on the backend.