lucene.net - search term too short??

Question

Hello i have an application with lucene, when i'm searching terms like "a", "a*", "an", "an*" ,...

throw an error:

Ausnahmedetails: Lucene.Net.Search.BooleanQuery+TooManyClauses: Systemfehler

Quellfehler:

Zeile 130:            
Zeile 131:            Dim searcher As IndexSearcher = New IndexSearcher(rootpath + "\" + index_root) 'Suche auf diesem Verzeichnis
Zeile 132:            Dim hits As Hits = searcher.Search(query)
Zeile 133:   
Zeile 134: 

but terms that contains three or more letters don't throw an error.

i'm really confused about that.

more code:

Public Sub lucene_search(ByVal strSuchbegriff As String)




        Dim parser As QueryParser
        Dim query As Query


        If (check_volltextsuche.Checked = True And check_dateinamensuche.Checked = False) Then

            parser = New QueryParser("bodytext", analyzer) 'bodytext=typfeld der durchsucht wird

            Try
                query = parser.Parse(strSuchbegriff)

            Catch

                meldung.Text = "Falsche Verwendung der Suchsyntax"
                query = parser.Parse("Suchsyntax")
                ItemsGrid.Visible = False
                myexception = True
            End Try


        ElseIf (check_volltextsuche.Checked = False And check_dateinamensuche.Checked = True) Then

            parser = New QueryParser("title", analyzer)

            Try
                query = parser.Parse(strSuchbegriff) '* um teile danach zu finden --> gesamten filename durchsuchen
            Catch

                meldung.Text = "Falsche Verwendung der Suchsyntax"
                query = parser.Parse("Suchsyntax")
                ItemsGrid.Visible = False
                myexception = True

            End Try


        Else



            parser = New MultiFieldQueryParser(New [String]() {"title", "bodytext"}, New StandardAnalyzer())


            Try
                query = parser.Parse(strSuchbegriff)

            Catch

                meldung.Text = "Falsche Verwendung der Suchsyntax"
                query = parser.Parse("Suchsyntax")
                ItemsGrid.Visible = False
                myexception = True
            End Try





        End If

        '################
        'Do the search ##
        '################

        If myexception = False Then




            Dim searcher As IndexSearcher = New IndexSearcher(rootpath + "\" + index_root) 'Suche auf diesem Verzeichnis
            Dim hits As Hits = searcher.Search(query) '<-- ERROR

thanks in advance :>

Answer 1

Are you sure it's failing for just "a" as well?

For the "a*" and "an*" cases, the reason this fails because Lucene turns that expression in a prefix search and basically turns it into a giant "OR" query with all of the terms defined in the index that start with "a" (or "an"). So if you have "aardvark", "antler", "animal", etc, then "a*" is the same as "aardvark OR antler OR animal OR ..."

Luncene also has a limit on the number of terms you can combine in an "OR" query, by default it's quite small (because too many terms can severely affect performance) and if there are too many terms, it will throw the BooleanQuery+TooManyClauses exception that you found.

You'll probably find that a query like "x*" or "qr*" does not throw an exception: this is because you (likely) don't have many terms that start with "x" or "qr".

To fix the problem you have a couple of options:

1. Refine the query further, simply don't allow single-letter prefix queries
2. Increase the maximum clause count by calling setMaxClauseCount first (I would try to avoid this, though, since it can affect performance as I said)