I'm experimenting with an idea using SQL Server's Full Text Indexing. It seems perfect for the task, but what my client wants is a very Google-like results summary, where the results display an extract of the text around their search term.
If I search for "house"...
My House Is a Very, Very, Very Fine House
...thanks for coming to visit our house today...don't you like this house...hey, why are you setting fire to my house?...
This isn't too hard if their search term is an exact match for what the search hit on. You could simply do some tedious parsing of the text to generate the extract.
But what happens with inflectionals and stemming? If I search for "walk," the query may hit on "walking," "walked," etc. I would need to know exactly what word inside the search result it hit on, so I would know from where to base my extraction.
This area seems ripe for some commercial product add-on, or perhaps there's an elegant way to do it that I'm not considering?
(And, yes, we are aware of the the GSA and the Google Mini. There are some subtle reasons why they might not work in this case, so we're trying SQL FTI first.)