How can I ftp multiple files?

Question

I have two unix servers in which I need to ftp some files. The directory structure is almost same except a slight difference, like:

server a                 server b
miabc/v11_0/a/b/c/*.c    miabc/v75_0/a/b/c/
miabc/v11_0/xy/*.h       miabc/v11_0/xy/

There are many modules:

miabc
mfabc

The directory structure inside them is same in both the servers except the 11_0 and 75_0. And directory structure in side different modules is different

How can I FTP all the files in all modules into the corresponding module in second server b by any of scripting languages like awk, Perl, shell, ksh using FTP?

Answer 1

I'd say if you want to go with Perl, you have to use Net::FTP.

Once, I needed a script that diffs a directory/file structure on an FTP server with a corresponding directory/file structure on a local harddisk, which lead me to write this script. I don't know if it is efficient or elegant, but you might find one or another idea in it.

hth / Rene

Answer 2

I suggest you could login on any of the server first and go to the appropraite path miabc/v75_0/a/b/c/ . From here you need to do a sftp to the other server.

1. sftp user@servername
2. Go to the appropraiate path which files needs to be transferred.
3. write the command mget *

Answer 3

I'd use a combination of Expect, lftp and a recursive function to walk the directory structure.

Answer 4

If the file system supports symlinking or hardlinking, I would use a simple wget to mirror the ftp server. in one of them when you're wgetting just hack the directory v11_0 to point to 75_0, wget won't know the difference.

server a:

1. go to /project/servera
2. wget the whole thing. (this should place them all in /project/servera/miabc/v11_0)

server b:

1. go to /project/serverb
2. create a directory /project/serverb/miabc/75_0, link it to /project/servera/v11_0:
   • ln -s /project/serverb/miabc/75_0 /project/servera/v11_0
3. wget serverb, this will be followed when wget tries to cwd into in 75_0 it will find itself in /project/servera/v11_0

Don't make the project harder than it needs to be: read the docs on wget, and ln. If wget doesn't follow symbolic links, file a bug report, and use a hard link if your FS supports it.

Answer 5

It sounds like you really want rsync instead. I'd try to avoid any programming in solving this problem.