How do I watch a folder for changes and when changes are done using Python?

Question

i need to watch a folder for incoming files. i did that with the following help:
http://stackoverflow.com/questions/182197/how-do-i-watch-a-file-for-changes-using-python
the problem is that the files that are being moved are pretty big (10gb) and i want to be notified when all files are done moving. i tried comparing the size of the folder every 20 seconds but the file shows its correct size even tough windows shows that it is still moving. i am using windows with python

i found a solution using open and waiting for an io exception. if the file is still being moved i get errno 13.

Answer 1

If you have control over the process of importing the files, I would put a lock file when starting to copy files in, and remove it when you are done. by lock file I mean a tmp empty file, which is just there to indicate that you are coping a file. then your py script can check for the existence of the lock files.

Answer 2

You may be able to use os.stat() to monitor the mtime of the file. However be aware that under various network conditions, the copy may stall momentarily and so the mtime is not updated for a few seconds, so you need to make allowance for this.

Another option is to try opening the file with exclusive read/write which should fail under windows if the file is still opened by the other process

Answer 3

The most reliable method would be to write your own program to move the files.

Answer 4

try checking for the last-modified time change instead of the filesize during your poll.

Answer 5

You should take a look at this link:

http://timgolden.me.uk/python/win32_how_do_i/watch_directory_for_changes.html

There you can see the comparison of the method you are speaking about (simple polling) with two other windows-specific techniques which, in my opinion, offers a really better solution to your problem!

Otherwise, if you are using linux, there's iNotify and the relative Python wrapper:

Pyinotify is a pure Python module used for monitoring filesystems events on Linux platforms through inotify

Here: http://trac.dbzteam.org/pyinotify