We have a C# Windows service polling a folder waiting for an FTP’ed file to be posted in. To avoid using the file when it is still being written to we attempt to get a lock on the file first, however, there seems to be occasions where we are getting a lock on the file after the FTP’ed file is created but before the file is written to, so we end up opening an empty file.
Is there a reliable anyway to tell if the FTP is complete?
I'm always a big fan of the .filepart protocol, so that no matter what transfer protocol you use (ftp,ssh,rsync,etc) you have the same understanding.
This isn't a direct answer to your question, but instead of searching for a ftp-only solution a more generic solution could be better for you in long run.
(.filepart: rename the file,test.txt to test.txt.filepart, then when it is done, name it back to test.txt)
You could possibly change the filename before upload, then rename it after it's done. that way it will look like it doesn't exist until finished.
Rather than polling, you might want to have a look at System.IO.FileSystemWatcher.
A practice I've seen done is you transfer two files, One which is the actual file, then a second one which will we can call a .done file. The ideal is as soon as you see the .done file you know the first file should be done.
Other options include watching the file for modifications and wait for a certain ammount of time of no modifications. Of course this is not full proof.
Kyle makes a good point that adding a checksum to the .done file and/or indicating the size of the first file is a good protection against fringe cases.
What about using a folder watcher to index the contents and if a files size does not change within 5 mins you can pretty-much guarantee the upload has been finished.
The time out could be tied to the timeout of your FTP server to.
http://www.codeproject.com/KB/files/MonitorFolderActivity.aspx
I've always used a checksum file. So you send a checksum file that denotes the filesize and the checksum. You'll know the file is uploaded correctly when the checksum in the first file matches the actual checksum on the file system.
The method I've used in the past is a mix of some of the other replies here. i.e. FTP a file using a different extension to the one expected (eg FILENAME.part) then rename it with the proper extension as the last step of uploading.
On the server, use a FileSystemWatcher to look for new files with the correct extension. The FSW will not see the file until it's renamed, and the renaming operation is atomic so the file will be complete and available the moment it's been renamed.
Renaming or moving files of course relies on you having control over the uploading process.
If you do not have any control over how the files are uploaded, you will be stuck with using the FSW to know a file is being uploaded, then monitoring it's size - when it's unchanged for a long period of time you may be able to assume it's complete.
Everybody, you really, really have to read the 'Tales from the interview' article on The Daily WTF, more precisely, the 2nd story ("I Guess That Would Work, Too")
It gives the answer to this question, you may learn a thing or two about interviews, and you'll probably laugh to tears - I know I did when I read it :-).