An API to use to retrieve similar words from a keyword?

Question

I'm writing a search engine in C#, retrieving rows from a SQL database. I'd like the search to also include similar words - for example, if a user searches for "investing", the search will also return matches for "investment", or if the user searches for "financial", the search will also return matches for "finance".

How can I retrieve similar words such as these from a search keyword?

Answer 1

What you're looking for is stemming. You may want to look at what's available in Lucene.net... although it's also possible that SQL Server supports this natively with full text indexing. Indeed, it looks like it, given this article.

Answer 2

What you're trying to accomplish is known as "Stemming". Read the Wikipedia article for more info:

http://en.wikipedia.org/wiki/Stemming

Answer 3

If you're using SQL Server you can take advantage of the FREETEXT search, which supports stemming:

Select * from SomeTable
where FREETEXT(*,'invest')

The above searches all columns for all forms of the word invest. It's equivalent to:

Select * from SomeTable
where CONTAINS(*,'"invest" or "invests" or "investor" 
                  or "investing'" or "invested" or "investor's" ... )

Here's an MSDN article with more examples and documentation.

Answer 4

Additionally, soundex searching can help find matches with similar phonetics. This is supported in SQL Server SOUNDEX() function. .NET doesn't appear the have it built-in, but CodeProject has several implementations.