How do I delete old documents from Lucene/Lucene.NET

views:

318

answers:

+2 Q:

How do I delete old documents from Lucene/Lucene.NET

What is the idiomatic way to delete old documents from a Lucene Index?

I have a date field (YYYYMMddhhmmss) on all of the documents, and I'd like to remove anything more than a day old (for example).

Should I perform a filtered search or enumerate through the IndexReader's documents?

I'm sure the question is the same regardless of which platform Lucene is running on.

Thanks!

+3 A:

Searching for YYYYMMdd* should work as currently dates are stored as text strings. Once you have the results, you could use IndexReader.delete to remove the docs you're not interested in. That seems to me the best way to achieve this.

synhershko 2009-10-17 17:38:33

One problem I see with that approach is that I'll get a "TooManyClauses" exception when there are more than old 1024 documents.

Eric Nicholson 2009-10-18 12:32:14

This really depends on your implementation. I will need to know the specifics, but as a general rule you could either remove this warning for those searches since they are maintenance only anyway (by setting a higher max clauses count), or make more specific searches (YYMMddhh* etc.). Again, all depends on your environment and implementation.

synhershko 2009-10-18 23:20:46

I ended up doing a slight variation of this, by using a MatchAllDocsQuery and a RangeFilter. Seems to be working OK so far...

Eric Nicholson 2009-10-20 15:11:39

+2 A:

You could try using low-level APIs of Lucene.

Get Term Enumerator from index with the term "YYYY". Iterate of the term enumerator to get terms. If the term's text doesn't with current date (or previous date), call IndexReader.deleteDocuments(term) with that term.

Since you are not using Query object, you will not get search related exception.

Shashikant Kore 2009-10-19 14:50:34

ansaurus

tags:

views:

answers:

How do I delete old documents from Lucene/Lucene.NET

related questions