are there best practices or tricks for indexing/monitoring a drive for files?

Question

I need to find and monitor all the photos on a hard drive or a folder for a photo organizer. Currently I'm doing this naively: recursively traversing, manually marking folders as indexed, and repeating that process to catch when photos are added or moved.

The problem is with a large enough folder tree this is very expensive, so I'm looking for tips to do this differently and/or tips on keeping it a low cpu process.

Ideally solutions would be not platform-dependent.

EDIT: I'm using xulrunner currently, but could compile a module do platform specific stuff.
What about the first run? Is there no solution (even platform-dependent) besides running through the entire folder tree manually.

Answer 1

I don't know of a way to do this in a platform independant way, but on Linux I'd hook into inotify to call something when a file gets added or updated. You could even use inotify-tools to run a script when that happens, so you don't have to be running all the time to capture all these events if they're infrequent. Just have the script update the database, and optionally notify your gallery/display program if it's running.

Answer 2

Are you coding on .NET? If so, you could use the FileSystemWatcher class instead.

Answer 3

Why not user a filewatcher program, which will notify you of changes in particular folder trees?

If you want to write your own you could use the FileSystemWatcher class to do it.

Answer 4

Ideally solutions would be not platform-dependant.

Impossible. The Win32API has FindFirstChangeNotification, Linux has inotify (and others), Mac OS X has FSEvents, et cetera. This is stuff that's very low-level, and no OS does it the same as any other OS. If you want something cross-platform, you have to find an API with several backends that works on the platforms you want, but if there are any of these, I haven't yet found them.