Indexing token bigrams in Lucene

Question

Hi, I need to index bi-grams of words (tokens) in Lucene. I can produce n-grams and than index them, but I am wondering if there is something in Lucene which will do this for me. I found out that Lucene indexes only n-gram of chars. Any ideas?

Answer 1

Depending on why you need to index bi-grams, SpanQuery and/or SnowballAnalyzer may be helpful.

Answer 2

Use the NGramTokenizer:

http://lucene.apache.org/java/2_3_2/api/contrib-analyzers/org/apache/lucene/analysis/ngram/NGramTokenizer.html