Copying files and dirs on remote server while excluding some of them

Answer 1

For this you should really be using rsync.

I tend to uye an rsync-exclude.txt file to specify what I don't want as it's more future proof.

/public_ftp/.ftpquota
/tmp
/var/local/backups/rsyncs
/backup/rsync
/proc
/dev

so a command could be

rsync --exclude-from rsync-exclude.txt --stats -avz -e ssh \
      --numeric-ids /syncfrom/dir  [email protected]:/backup/sync-to-dir

edit::

In the case of a local server you can still use rsync, however you could also use tar and exclude what you don't want.

 (cd dir1;tar --exclude 'web2/*' -cf -) | (cd dir2; tar -xvf -)

or find dir1 dir2 >exclude-files (cd dir1;tar --exclude-from exclude-files -cf -) | (cd dir2; tar -xvf -)

Answer 2

I do the same thing when deploying new releases to my webserver. Is it possible for you to use rsync over ssh? rsync allows you to specify an --exclude option and specify either the dirs/files on the command line or via a file. Here's a pretty good writeup that I've used in the past: http://articles.slicehost.com/2007/10/10/rsync-exclude-files-and-folders

Answer 3

Hello llaskin,

Since what you want to do is "copy all files except those", following your example you could do this :

 ssh server2 "cp -rv /var/www/!(web1|web3|file1.php|file3.php) /tmp"

But remember that this is very ugly to backup your server :p

Answer 4

The first two poster's both have good suggestions about rsync. Here's a more complete outline of the process.

(1) You want to backup server 2 before you sync from server 1, so let's do that with rsync. Here's the command as seen from server 1 (assuming it has access to server 2):

ssh user@server2 "rsync $RSYNC_OPTS /var/www/ /path/to/backup"

(2) With server 2 backed up, let's now sync from server 1 (again, as seen from server 1)

rsync $RSYNC_OPTS /path/to/www/ user@server2:/var/www/

As long as you use sane RSYNC_OPTS, the backup and sync should both be reasonable. Richard had a reasonable suggestion for the options:

RSYNC_OPTS="--exclude-from rsync-exclude.txt --stats -avz --numeric-ids -e ssh"

If you want an accurate reproduction, I'd recommend --delete or --delete-after as well. Be sure to lookup details on any options you're unfamiliar with.