Is there a way to generate separate index files from a single one based on some sort of rule without reindexing the documents again?
The original index contains none-stored fields which means I can't read documents and add them to the destination index.
One option mentioned in SO is to clone the index into many and then delete the documents that don't belong to that index. I'm looking for a better solution.
One option mentioned in SO is to clone the index into many and then delete the documents that don't belong to that index. I'm looking for a better solution.
What's wrong with this solution? This strikes me as a very clean solution, involving just a few lines of code.
UPDATE:
Regarding the scenario where you have a 100G index, wanting to split 500 times, try this: for every subset of documents that you want to carve out of the index, create hard links to the source index, open the linked index and delete the documents that don't belong to that index. If you're on Linux, hard linking the directory can be done by:
cp -lrp myindex myindex.copy
This can be done as many times as you need to and the links do not consume any disk space.