How to determine if a file has finished transferring to be able to copy.

Question

What is the best method to make sure that a file that is transferred via ftp has finished before another process attempts to move/copy that file?

Answer 1

1) If you are in control of the sending ; once you have sent the actual file, ftp a control empty file with the same name but different .ext. Test for this control file and the actual file will be okay.

2) As an extra test; check the file age and wait a further 30 seconds before processing, this may be necessary to avert latency on network file locks.

Code Sample to explain 2)

const
  LastWriteAccessDwell = 30; // seconds
var
  SR: TSearchRec;
  LastAccess: TDateTime;

...
if LastWriteWithin( SR.FindData.ftLastWriteTime, LastWriteAccessDwell, LastAccess ) then
begin
  WriteText( format('Last Access @%s was less than %d seconds.', [DateTimeToStr(LastAccess), LastWriteAccessDwell]) );
  continue;
end;

function LastWriteWithin( FileTime: TFileTime; const Seconds: Cardinal; out LastAccess: TDateTime ): boolean;
var
  LocalFileTime: TFileTime;
  iFileAge: integer;
begin
  if Not( FileTimeToLocalFileTime( FileTime, LocalFileTime )
      and FileTimeToDosDateTime(LocalFileTime, LongRec(iFileAge).Hi, LongRec(iFileAge).Lo) ) then
    RaiseLastOSError;

  LastAccess:= FileDateToDateTime(iFileAge);
  result:= WithinPastSeconds( Now, LastAccess, Seconds );
end;

Answer 2

How about the ftp server? Maybe a plugin for the server that will fire an event whenever a positive completion reply (2xx Server Return) had been received when a file transfer is complete.

Sending files via ftp can be interrupted and paused for unknown amounts of time and you can never assume they are finished. I think the only way to be positive of completion ( or even file transfer without error ) is thru the ftp server.

Answer 3

Not sure how your FTP server works, but I'd periodically check to see if you can open the file exclusively as it's likely open for writing during the file transfer process and closed when finished.

Answer 4

I always used to do the following;

a) get the size of the file, in bytes
b) append this to the filename eg MyFile.Dat.16384
c) send the file over ftp etc

at the other end, you have a program watching the FTP incoming folder. Every second or so (adjust to suit), you scan the folder for files and when you find a file whose filesize matches the string at the end of the filename, you can be reasonably sure it's arrived and is free to be moved, renamed, whatever.

We built some logic into the scanning such that a file wasn't pounced on until the second time it had turned up in a scan, just to ensure that last second or so of possible processing/unlocking on the part of the FTP server.

This is fairly straightforward with Delphi, it's really just a findfirst/findnext/findclose type structure in a loop, running on a timer. Switch the timer off when you start a scan and restart it when you're done.