Search subset of objects using Compass/Lucene

Question

Hi,

I'm using the searchable plugin for Grails (which provides an API for Compass, which is itself an API over Lucene). I have an Order class that I would like to search but, I don't want to search all the instances of Order, just a subset of them. Something like this:

// This is a Hibernate/GORM call
List<Order> searchableOrders = Customer.findAllByName("Bob").orders

// Now search only these orders with the searchable plugin - something like
searchableOrders.search("name: foo")

In reality the relational query to get the searchableOrders is more complex than this, so I can't do the entire query (Hibernate + compass) in compass alone. Is there a way to search only a subject of instances of a particular class using Compass/Lucene.

Answer 1

Two ways of doing this:

The easiest from the implementation standpoing is do two searches (one findAll and search) on all objects and then find intersection between them. If you cache the result of findAll call, then you are really down to one query you have to make.

A more "clean" way to do this is to make sure to index the IDs of the domain objects with Searchable, and when you get the findAll result, pass in those IDs into the search query, thus limiting it.

I don't remember the Lucene syntax off the top of my head, but you'd have to do something like

searchableOrders.search("name: foo AND (ID:4 or ID:5 or ID:8 ...)" )

You may run into query size limits in Lucene, but I think there are settings that allows you to control query length.

Answer 2

One way to do this is with a custom Filter. For example, if you wanted to filter based on ids for your domain class, you would add the id to the searchable configuration for the domain class:

static searchable = {
   id name: "id"
}

Then you would write your custom filter (which can go in [project]/src/java):

import org.apache.lucene.search.Filter;
import java.util.BitSet;
import org.apache.lucene.index.TermDocs;
import org.apache.lucene.index.Term;
import org.apache.lucene.index.IndexReader;

import java.io.IOException;
import java.util.List;

public class IdFilter extends Filter {

    private List<String> ids;

    public IdFilter(List<String> ids) {
        this.ids = ids;
    }

    public BitSet bits(IndexReader reader) throws IOException {
        BitSet bits = new BitSet(reader.maxDoc());
        int[] docs = new int[1];
        int[] freqs = new int[1];
        for( String id : ids ) {
            if (id != null) {
                TermDocs termDocs = reader.termDocs(new Term("id", id ) );
                int count = termDocs.read(docs, freqs);
                if (count == 1) {
                    bits.set(docs[0]);
                }
            }
        }
        return bits;
    }
}

Then you would put the filter as an argument to your search (making sure to import the Filter class if its in a different package):

def theSearchResult = MyDomainClass.search( 
{
    must( queryString(params.q) )       
}, 
params,
filter: new IdFilter( [ "1" ] ))

Here I'm just creating a hard-coded list with a single value of "1" in it, but you could retrieve a list of ids from the database, from a previous search, or wherever.

You could easily abstract the filter I have to take the term name in the constructor, then pass in "name" like you want.