I am working on an indexing/searching feature of a site. The indexing runs every 6 hours and takes about 2 minutes.
What is the best approach to forbid the reading from the indexes folder during these 2 minutes?
I am working on an indexing/searching feature of a site. The indexing runs every 6 hours and takes about 2 minutes.
What is the best approach to forbid the reading from the indexes folder during these 2 minutes?
Why not build the indices in another (perhaps temporary) directory, and then copy the new indices over the old ones when the indexer is finished? Then you cut the time during which the indexes are not valid to the length of time it takes the files to copy.
edit: introduced the locking strategies below
Without knowing more about how the processes that use the index use it, it's hard to suggest a good locking strategy. If the indices are acquired, read, and released quickly and you've got exclusive control over the index directory, you could write a lockfile to the index directory (something like indexlock.donotuse
) at the start of the index copy, and remove it when you're done. If references to the indices are held for a long time, the lockfile approach will still work but you have more complex coordination to do.
You could also store index-versioning metadata in the index itself, and place the updated index next to the old one. Then, when you go to read from an index, check whether a newer one exists before using the one you've got.