Solr search for lots of values

Question

I'm using Solr to search for a long list of IDs like so:

ID:("4d0dbdd9-d6e1-b3a4-490a-6a9d98e276be"
    "4954d037-f2ee-8c54-c14e-fa705af9a316"
    "0795e3d5-1676-a3d4-2103-45ce37a4fb2c"
    "3e4c790f-5924-37b4-9d41-bca2781892ec"
    "ae30e57e-1012-d354-15fb-5f77834f23a9"
    "7bdf6790-de0c-ae04-3539-4cce5c3fa1ff"
    "b350840f-6e53-9da4-f5c2-dc5029fa4b64"
    "fd01eb56-bc4c-a444-89aa-dc92fdfd3242"
    "4afb2c66-cec9-8b84-8988-dc52964795c2"
    "73882c65-1c5b-b3c4-0ded-cf561be07021"
    "5712422c-12f8-ece4-0510-8f9d25055dd9"...etc

This works up to a point, but above a certain size fails with the message: too many boolean clauses. You can increase the limit in solrconfig.xml, but this will only take it so far - and I expect the limit is there for a reason:

<maxBooleanClauses>1024</maxBooleanClauses>

I could split the query into several little ones, but that would prevent me then sorting the results. There must be a more appropriate way of doing this?

Answer 1

You should be using a Lucene filter instead of building up the huge boolean query. Try using FieldCacheTermsFilter and pass that filter in to your Searcher. FieldCacheTermsFilter will translate your UID's to a Lucene DocIdSet, and it'll do it fast since it's doing it via the FieldCache.