Too Many open file exception while indexin using solr

Question

Hi,

I am using SOLR for indexing documents in my web application and solr.war is deployed on the jboss server. But while indexing i am getting too many files open exception. Below is some of exceptions stack trace:

12:31:33,267 ERROR [STDERR] Exception in thread "Lucene Merge Thread #0"
12:31:33,267 ERROR [STDERR] org.apache.lucene.index.MergePolicy$MergeException: java.io.FileNotFoundException: /data/jbossesb/bin/solr/data/index/_2rw.prx (Too many open files)
12:31:33,267 ERROR [STDERR] at org.apache.lucene.index.ConcurrentMergeScheduler.handleMergeException(ConcurrentMergeScheduler.java:351)
12:31:33,267 ERROR [STDERR] at org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:315)
12:31:33,267 ERROR [STDERR] Caused by: java.io.FileNotFoundException: /data/jbossesb/bin/solr/data/index/_2rw.prx (Too many open files)
12:31:33,267 ERROR [STDERR] at java.io.RandomAccessFile.open(Native Method)

Answer 1

File Descriptor will be your most likely cause.

Check the limit which your operating system has set. and adjust accordingliy. on Unix, the command to view and set is ulimit.

Answer 2

As explained in this SOLR Jira, you can try the following options:

• increasing your ulimit using: ulimit -n 1000000
• set useCompoundFile to true in solrconfig.xml to use Lucene's compound file format
• use a lower mergeFactor which will result in fewer segments and hence fewer open files.

Answer 3

Optimize the index. It probably has too many segments.

Answer 4

Also try reducing the merge factor