Why size of lucene index increased if i index the same data?

Question

I implemented Hibernate search in my application i.e. based on Lucene. Whenever, i indexes the database, the size of the lucene indexes increase. But, the result of the query return same no of results every time.

Why the size of lucene increases each time if i index the same data everytime?

FullTextSession fullTextSession = Search.getFullTextSession(getSession());
 org.hibernate.Transaction tx = fullTextSession.beginTransaction();

 Criteria criteria = fullTextSession.createCriteria(getPersistentClass())
    .setResultTransformer(CriteriaSpecification.DISTINCT_ROOT_ENTITY)
    .setCacheMode(CacheMode.IGNORE)
    .setFetchSize(pageSize)
    .setFlushMode(FlushMode.MANUAL);


    int i = 0;
    List<ProdAttrAssociationVO> results = null;
    do {
      criteria = criteria.setFirstResult(i)
        .setMaxResults(pageSize);
      results = criteria.list();

      for (ProdAttrAssociationVO entity : results) {
        fullTextSession.delete(entity);
        fullTextSession.index(entity);
   }

      // flush the index changes to disk so we don't hold until a commit
      if (i % batchSize == 0) {
        fullTextSession.flushToIndexes();
        fullTextSession.clear();
      }

      i += pageSize;
    } while (results.size() > 0);


 System.out.println("ProdAttrAssociation Indexing Completed");
 tx.commit();

Answer 1

I don't know anything about Hibernate, but generally in Lucene, deleted documents stay on the index until it is optimized. That could explain why you're seeing the index only growing.

Try to run optimize() on the index. Not sure how you do it from Hibernate (I see it's a method on SearchFactory).

Hope this helps.