How can I write a bat file to download only new FTP files?

Question

Hi,

I would like to use the built in windows ftp application to download some files on a periodic basis using a scheduled task and a .bat file.

However I don't want to download the whole directory each time just the files that have changed or are new.

Can this be done or do I need to write .NET application or the like?

UPDATE: After bashing my head against mirror.pl for several hours yesterday and failing to get it to work. I had a hunt around for other programmes which might be similar. I finally settled on scriptFTP

It was simple and easy to get up and running. However I have accepted mirror.pl as he answer as 1. It pointed me in the right direction (not rolling my own!) and 2. I'm sure it works a treat for those who can get past its trickiness!

Answer 1

You won't be able to do this with just the ftp client. I have a script which does the opposite to you - it uploads items (so I don't have to worry about keeping it sync'd properly).

There are two other things you can try. Try the 'rsync' program - this will solve all your synchronisation problems:

http://samba.anu.edu.au/rsync/

Alternatively, look at using a Powershell script to do it for you. What you would have to do is parse the datetime given by the ls command and then compare it to the datetime that you have on your local system. It's worth noting that different ftp servers have different standards for returning datetime info, so if you run it on more than one server, this will have to be a consideration.

Answer 2

G'day,

Don't even try to roll your own.

We use mirror.pl which is a Perl script written by Lee McLoughlin.

An excellent utility that will do what you want.

cheers,

Rob

Edit: Yes. I know the doc's can be a bit obstruse but it is well worth getting the ducks in a row. Once you understand some basic, unfortunately undocumented, concepts mirror is really easy to use.

Answer 3

Yes, there is a way to create FTP Scripts and use it on your windows using the command:

ftp -s:script.ftp ftp.domain.com

Here you go an example:

user myuser mypass
bin
cd /dir
get file.zip
bye

That way you can use commands to read files from FTP and with a little Batch/FTP magic you can compare dates and time to know what you should download:

More commands at http://www.robvanderwoude.com/ftp.html

Answer 4

'rsync' is an excellent utility. Alternatively, you can also use 'curl'. The man page documents how to download only new files in the "Time Conditions" section.

I would not recommend FTP scripts because there is no fault tolerance if the file is missing or connection is dropped.

Answer 5

PyroBatchFTP will perform this task for you, both uploads and downloads. Its script driven and can be called from a scheduled task, command line, includes code samples for VB/VBA/C++/Powerbuilder app or it has its own internal scheduler. It can also be run as a windows service.

The commands to use are:

GetSync -rnm "*"

(Get all newer or non-existing files from server)

GetSync -nmy24 "*"

(Get all files from the server which were modified within the last 24 hours)

Full script might be:

Connect "ftp.ger.emtec.com", "anonymous", "[email protected]"
LocalChDir "C:\TEMP"
RemoteChDir "pyrotrans/batchftpdemo"
GetSync -rnm "*"
Disconnect

Answer 6

scriptFTP seems a great choice if you are on Windows. Not only does it have a full-featured "SYNC" command, it also supports SFTP, which many of the alternatives mentioned don't (including mirror.pl and PyroBatchFTP)

Answer 7

All though not free. 4NT now Take Command have a built in FTP command so that you can actually run command line commands like dir ftp:*.*. All of the batch commands and 4NT commands work with using FTP as a drive letter. 4NT/Take Command enhance all of the normal batch commands so that you can filter/select on things like file sizes, date/time, etc.

Answer 8

you should use time (local time) for comparing files then if they differ try to download or upload i manage my site by this method and it work easily visit my site in the download center i have shared my own written script you can download and use it http://ehrajat.com