.Net FileWatcher fails for ~80+ files

Question

I'm using .net 2.0 filewatcher to watch a folder for new files. It works perfectly except when I put more than ~80 files at once. The event just doesn't trigger anymore. It's as if the filewatcher is set to keep track of certain number of files.

For the time being I have asked the user not to put more than 50 files at a time and that seems to work however I would like to fix it so that hundreds of files can be dropped into the folder at once.

Here's the code I'm using for the event. It's pretty standard stuff nothing fancy.

FileWatcher = new FileSystemWatcher();
FileWatcher.Path = ConfigurationManager.AppSettings["FolderOfFilesToWatch"];
FileWatcher.NotifyFilter = NotifyFilters.FileName;
FileWatcher.Filter = "*_*_*.*";
FileWatcher.Created += new FileSystemEventHandler(watcher_Created);
FileWatcher.EnableRaisingEvents = true;


static void watcher_Created(object sender, FileSystemEventArgs e)
{
Console.Write(e.Name);
}

Any ideas?

Answer 1

You probably need to increase the FileSystemWatcher.InternalBufferSize. By default, FileSystemWatcher uses a smaller buffer for performance, and can overflow if too many changes occur in a short time frame.

Try setting a larger buffer size to prevent that from occurring.

Answer 2

See Considerations for File Changes on High-Volume Systems.

Answer 3

I use FileWatcher but I took a "belt and suspenders" approach. Whenever I get a FileWatcher event I stick around and check for files I haven't seen before (in my case I have a file catalog). Believe me, there will be a time when you get thousands of files dropped in a folder and you must have another safeguard to account for all of them.

Another alterative you may look into is change journals, although it is (AFAIK) limited to disks attached to your machine, whereas you can use FileWatcher to watch UNC paths as well.

Answer 4

It might be possible that you are exceeding your system's ThreadPool count. When The FileSystemWatcher fires off an event it does so from a thread retrieved from this pool. It is possible lose these events if you don't have enough threads allocated to your ThreadPool.

http://msdn.microsoft.com/en-us/library/system.threading.threadpool.aspx

As others, especially, 'ocdecio' have noted it is good not to completely trust the FileSystemWatcher. It is notoriously inconsistent under heavy load.