tags:

views:

51

answers:

2

I'm trying to sort a solr query by a field ignoring stopwords, but can't seem to find a way to do that. For example, I want the results to be sorted like:

  • Charlie
  • A Fox
  • Helicopter

Is this possible? Right now the field type is defined like:

<fieldType name="alphaOnlySort" class="solr.TextField" sortMissingLast="true" omitNorms="true">
  <analyzer>
    <tokenizer class="solr.KeywordTokenizerFactory"/>
    <filter class="solr.LowerCaseFilterFactory" />
    <filter class="solr.TrimFilterFactory" />
    <filter class="solr.StopFilterFactory" words="stopwords.txt" ignoreCase="true"/>
    <charFilter class="solr.MappingCharFilterFactory" mapping="mapping-ISOLatin1Accent.txt"/>
    <filter class="solr.PatternReplaceFilterFactory" pattern="([^a-z])" replacement="" replace="all" />
  </analyzer>
</fieldType>

And the field is added like:

<field name="title" type="alphaOnlySort" indexed="true" stored="false"/>

It seems like someone else would've had to do this too? Or is sorting without stopwords a no-no?

A: 

You need to actually add the Stopwords Filter to the chain of parsers. Paste your text to be indexed into the field analyser in Solr Admin and you'll see that the A in A Fox is not being dropped!

Eric Pugh
I have the stopword filter in there, but it wasn't showing up in the SO question. I've fixed it. I also didn't realize there was a field analyser. I can use that for much quicker debugging, but I'm still having the issue... The only thing I can think of is the stopwords.txt file needs to be somewhere else?
Jamie
stopwords.txt should be in your /conf/ dir
Eric Pugh
A: 

Using the analyser mentioned by Eric, I've determined that the stop word filter only grabs exact words that matched, not pieces of a sentence. So, if there's a token of "THE" it will remove it. But, if there's a token of "THE FISH", it won't touch it.

So, is there a way to make this work? I just want to sort on a field, ignoring any stopwords. But the result is a bunch of sentences (or book names).

Jamie