XCopy - Only grab files that are fully uploaded

Question

I have an automated job that pulls files that are uploaded to our servers via a client facing site using xcopy.

Is there any way to only pull files that are fully uploaded?

I have thought about creating a second "inProcess" folder that will be used for uploading and then move those files once fully uploaded, but that still creates a window of time when the file is in transition to a "Done" folder...

Any thoughts?

Answer 1

use the .filepart extension for temporary files.

It's probably the most simple and clear way of doing this.

WinSCP does this.

Answer 2

You can upload an MD5 hash of the file and then upload the file and if the file uploaded doesn't match the MD5 then it's not finished (or if it takes to long, perhaps it didn't upload properly)

MD5 is often used to check the integrity of a file by creating a hash that represents the file. If the file varies at all, it will almost always (as in, basically never for our purposes) generate a different MD5 hash. The only reason a file would not match it's previously uploaded MD5 hash is if it wasn't finished or the MD5/file was corrupted on upload.

There is also this. but it's perl and from expert exchange (ick)