views:

376

answers:

3

I'm trying to mirror files on FTP server. Those files can be very large so downloads might be interrupted. I'd like to keep the original files while downloading partial files to a temporary folder and once completed override local older versions.

  1. Can I do this? how?
  2. Is there another easy to use (command line) tool that I can use?
A: 

I'm not sure if I understood correctly, but there is -c option that allows resuming downloads. Bear in mind that your FTP server must also support it.

Anonymous
A: 

Looking at the wget manual I can't see this functionality, however you could write a bash script to do what you want, which would essentially run an individual wget for each file then move it using normal mv command.

Alternativly have a look at rsync according to the manual there is a paramater that sets a temp dir

-T  --temp-dir=DIR          create temporary files in directory DIR

I am not 100% sure wheather this is where it puts the files during downloads as not had chance to try it out.

Mark Davidson
+1  A: 

First, download the files to a temp directory. Use -c so you can resume.

After the download has completed, use copy, rename or rsync to copy the files to the final place.

Note: Consider using rsync for the whole process because it was designed for just this use case and it will cause much less strain on the server and the Internet. Most site admins are happy if you ask them for an rsync access just for this reason.

Aaron Digulla