rsync --delete --files-from=list / dest/ does not delete unwanted files

Question

Hi,

as you can see in the title i try to sync a folder with a list of files. I hoped that this command would delete all files in dest/ that are not on the list, but it didn't.

So i searched a little bit and know now, that rsync can't do this.

But i need it, so do you know any way to do it?

PS: The list is created by a python script, so it is imaginable that your solution use some python code.

EDIT, let's be concrete:

The list looks like this:

/home/max/Musik/Coldplay/Parachutes/Trouble.mp3
/home/max/Musik/Coldplay/Parachutes/Yellow.mp3
/home/max/Musik/Coldplay/A Rush of Blood to the Head/Warning Sign.mp3
/home/max/Musik/Coldplay/A Rush of B-Sides to Your Head/Help Is Around the Corner.mp3
/home/max/Musik/Coldplay/B-Sides (disc 3)/Bigger Stronger.mp3

and the command like this:

rsync --delete --files-from=/tmp/list / /home/max/Desktop/foobar/

This works, but if i delete a line, it is not deleted in foobar/.

EDIT 2:

rsync -r --include-from=/tmp/list --exclude=* --delete-excluded / /home/max/Desktop/foobar/

That work neither ...

Answer 1

rsync is ideal for keeping directories in sync, among other useful things. If you do have an exact copy on the SOURCE, and want to delete files on the DEST, you can delete them from SOURCE and the rsync --delete option will delete them from DEST also.

However, if you just have an arbitrary list of files you want to delete, I suggest you use SSH to accomplish that:

ssh [email protected] rm /path/to/file1 /path/to/file2

This will execute the rm command on the remote host.

Using python, you could:

import subprocess
FileList = ['/path/to/file1', '/path/to/file2']
subprocess.call(['ssh', '[email protected]', 'rm'] + FileList)

~enjoy

Answer 2

Perhaps you could do this using a list of include patterns instead, and use --delete-excluded (which does as the name suggests)? Something like:

rsync -r --include-from=<patternlistfile> --exclude=* --delete-excluded / dest/

If filenames are likely to contain wildcard characters (*, ? and [) then you may need to modify the Python to escape them:

re.sub("([[*?])", r"\\\1", "abc[def*ghi?klm")

Edit: Pattern-based matching works slightly differently to --files-from in that rsync won't recurse into directories that match the exclude pattern, for reasons of efficiency. So if your files are in /some/dir and /some/other/dir then your pattern file needs to look like:

/some/
/some/dir/
/some/dir/file1
/some/dir/file2
/some/other/
/some/other/dir/
/some/other/dir/file3
...

Alternatively, if all files are in the same directory then you could rewrite the command slightly:

rsync -r --include-from=<patternlistfile> --exclude=* --delete-excluded /some/dir/ dest/

and then your patterns become:

/file1
/file2

Edit: Thinking about it, you could include all directories with one pattern:

/**/

but then you'd end up with the entire directory tree in dest/ which probably isn't what you want. But combining it with -m (which prunes empty directories) should solve that - so the command ends up something like:

rsync -m -r --delete-excluded --include-from=<patternfile> --exclude=* / dest/

and the pattern file:

/**/
/some/dir/file1
/some/other/dir/file3