What is the best way to select a text portion to cut based on keywords?

Question

When you search something in Stackoverflow it cuts the portion of the question description that best matches your criteria and after that it marks the criteria words.

I wonder the best way to do this manually in C#, meaning without the help of a full-text search engine.

The main problem is how to select the best text portion in a fast way?

What I did so far is:

1. I obtain the space indexes of the text. This allows me to know where the words begin so that I can start my substring tests from them.
2. From each of the space indexes, I get 300 characters ahead and test how many occurrences of the keywords I find.
3. I assume that the 300 characters long portion that has the most occurrences is the best so I cut it from the original text.

Is this a good approach? Is there a faster way? Is counting the number of occurrences the best way to find the most relevant portion?

Answer 1

Using this approach you will often find a best match with keywords near the start or end of the match, which means you won't have much context for those keywords. I'd add an extra condition that there must be n words on either side of keywords near the start and end of the match.

You could consider breaking the match at more convenient places, such as punctuation or conjunction words instead of spaces.

You might also want to look into term frequency - inverse document frequency to give different weightings to the keywords rather than just counting them.