Solr updates are very slow

Question

I am seeing extremely slow Solr updates in my database. The database only has 900 documents. We use autocommit with the following settings, and once in a while autocommit is taking long time blocking updates:

<autoCommit> 
  <maxDocs>10000</maxDocs>
  <maxTime>1000</maxTime>
</autoCommit>

What in the world can be happening for 74 seconds with 900 documents.

Ideas?

Here is the log snippet:

Oct 18, 2010 11:52:46 AM org.apache.solr.core.SolrCore execute INFO: [] webapp=/solr path=/update params={} status=0 QTime=59569 Oct 18, 2010 11:52:46 AM org.apache.solr.update.SolrIndexWriter getDirectory

Oct 18, 2010 11:53:21 AM org.apache.solr.core.SolrCore execute INFO: [] webapp=/solr path=/update params={} status=0 QTime=33586 Oct 18, 2010 11:53:21 AM org.apache.solr.update.processor.LogUpdateProcessor finish

Oct 18, 2010 11:54:40 AM org.apache.solr.core.SolrCore execute INFO: [] webapp=/solr path=/update params={} status=0 QTime=76098 Oct 18, 2010 11:54:41 AM org.apache.solr.update.DirectUpdateHandler2 commit

And the commit log:

Oct 18, 2010 11:54:00 AM org.apache.solr.update.DirectUpdateHandler2 commit

INFO: start commit(optimize=false,waitFlush=true,waitSearcher=true)

Oct 18, 2010 11:54:00 AM org.apache.solr.search.SolrIndexSearcher

INFO: Opening Searcher@29b003 main

Oct 18, 2010 11:54:00 AM org.apache.solr.update.DirectUpdateHandler2 commit

INFO: end_commit_flush

Oct 18, 2010 11:54:00 AM org.apache.solr.search.SolrIndexSearcher warm

INFO: autowarming Searcher@29b003 main from Searcher@718c93 main filterCache{lookups=0,hits=0,hitratio=0.00,inserts=512,evictions=0,size=257,warmupTime=19294,cumulative_lookups=3330661,cumulative_hits=12 5437,cumulative_hitratio=0.03,cumulative_inserts=3207537,cumulative_evictions=3184094}

Oct 18, 2010 11:54:20 AM org.apache.solr.search.SolrIndexSearcher warm

INFO: autowarming result for Searcher@29b003 main filterCache{lookups=0,hits=0,hitratio=0.00,inserts=256,evictions=0,size=256,warmupTime=19739,cumulative_lookups=3330661,cumulative_hits=12 5437,cumulative_hitratio=0.03,cumulative_inserts=3207537,cumulative_evictions=3184094}

Oct 18, 2010 11:54:20 AM org.apache.solr.search.SolrIndexSearcher warm

INFO: autowarming Searcher@29b003 main from Searcher@718c93 main queryResultCache{lookups=0,hits=0,hitratio=0.00,inserts=256,evictions=0,size=256,warmupTime=18604,cumulative_lookups=3084,cumulative_hits= 996,cumulative_hitratio=0.32,cumulative_inserts=2313,cumulative_evictions=0}

Oct 18, 2010 11:54:40 AM org.apache.solr.search.SolrIndexSearcher warm

INFO: autowarming result for Searcher@29b003 main queryResultCache{lookups=0,hits=0,hitratio=0.00,inserts=256,evictions=0,size=256,warmupTime=19925,cumulative_lookups=3084,cumulative_hits= 996,cumulative_hitratio=0.32,cumulative_inserts=2313,cumulative_evictions=0}

Oct 18, 2010 11:54:40 AM org.apache.solr.search.SolrIndexSearcher warm

INFO: autowarming Searcher@29b003 main from Searcher@718c93 main documentCache{lookups=0,hits=0,hitratio=0.00,inserts=0,evictions=0,size=0,warmupTime=0,cumulative_lookups=41846,cumulative_hits=33712,cumu lative_hitratio=0.80,cumulative_inserts=8134,cumulative_evictions=0}

Oct 18, 2010 11:54:40 AM org.apache.solr.search.SolrIndexSearcher warm

INFO: autowarming result for Searcher@29b003 main documentCache{lookups=0,hits=0,hitratio=0.00,inserts=0,evictions=0,size=0,warmupTime=0,cumulative_lookups=41846,cumulative_hits=33712,cumu lative_hitratio=0.80,cumulative_inserts=8134,cumulative_evictions=0}

Oct 18, 2010 11:54:40 AM org.apache.solr.core.SolrCore registerSearcher

INFO: [] Registered new searcher Searcher@29b003 main

Oct 18, 2010 11:54:40 AM org.apache.solr.search.SolrIndexSearcher close

Answer 1

With <maxTime>1000</maxTime>, you will have a commit at each second. Each time Solr does a commit, there is a couple of things that happens: the index searcher is closed and reopened, the caches are warmed with the old caches and some queries might be launch automatically (look at 'newSearcher' settings in your solrconfig). At some point, you might have overlapping commit. Solr will be usually real slow if this happen.

So ask yourself if you really need to have a commit at each second. If yes, it is probably better to remove all queries from the 'newSearcher', since committing that frequently renders them useless.

Answer 2

Another option would be, rather than depending on XML, Keep control with yourself. Have a timer or some criteria say (after adding 10 documents) etc and do commit by yourself. If you are using solrj then call solrserver.commit.