Extract key sentences from a text

Question

Hi,

do you know about an effective method for extracting key sentences from a text with their frequency parameters, etc and that can also do "stemming" (search also for similar sentences) ?

I wonder also if there is some software implementation

Thanks a lot

Answer 1

There are many good natural language processing (NLP) toolkits which will do sentence identification and extraction.

See http://en.wikipedia.org/wiki/Natural%5Flanguage%5Fprocessing%5Ftoolkits for a list.

Your requirement for "similar sentences" needs clarification. Maybe you are looking for a treebank (http://en.wikipedia.org/wiki/Treebank) where sentences are parsed and compared.

Answer 2

Here's a list of Peter Turney's papers about extracting keyphrases from text. Here's a list of Text Summarization Resources.Stemming and Text Similarity are different, large and evolving research areas. You have to give us a much more focused question in order to get a specific answer.

Answer 3

You might want to consider SumBasic. It's a surprising effective and simple algorithm that selects the key sentences based on the word frequencies. I don't know of any NLP toolkit that has it already implemented though.