Shell command to tar directory excluding certain files/folders

Question

Is there a simple shell command/script that supports excluding certain files/folders from being archived?

I have a directory that need to be archived with a sub directory that has a number of very large files I do not need to backup.

Not quite solutions:

The tar --exclude=PATTERN command matches the given pattern and excludes those files, but I need specific files & folders to be ignored (full file path), otherwise valid files might be excluded.

I could also use the find command to create a list of files and exclude the ones I don't want to archive and pass the list to tar, but that only works with for a small amount of files. I have tens of thousands.

I'm beginning to think the only solution is to create a file with a list of files/folders to be excluded, then use rsync with '--exclude-from=file' to copy all the files to a tmp directory, and then use tar to archive that directory.

Can anybody think of a better/more efficient solution?

EDIT: cma's solution works well. The big gotcha is that the --exclude='./folder' MUST be at the beginning of the tar command. Full command (cd first, so backup is relative to that directory):

cd /folder_to_backup
tar --exclude='./folder' --exclude='./upload/folder2' -zcvf /backup/filename.tgz .

Answer 1

Use the find command in conjunction with the tar append (-r) option. This way you can add files to an existing tar in a single step, instead of a two pass solution (create list of files, create tar).

find /dir/dir -prune ... -o etc etc.... -exec tar rvf ~/tarfile.tar {} \;

Answer 2

Your best bet is to use find with tar, via xargs (to handle the large number of arguments). For example:

find / -print0 | xargs -0 tar cjf tarfile.tar.bz2

Answer 3

You can exclude directories with --exclude for tar. If you want to archive everything except /usr you can use

tar -zcvf /all.tgz / --exclude=/usr

In your case perhaps something like

tar -zcvf archive.tgz arc_dir --exclude=dir/ignore_this_dir

Answer 4

You can have multiple exclude options for tar so

$ tar --exclude='file1' --exclude='patter*' --exclude='file2'

etc will work

Answer 5

You can use cpio(1) to create tar files. cpio takes the files to archive on stdin, so if you've already figured out the find command you want to use to select the files the archive, pipe it into cpio to create the tar file:

find ... | cpio -o -H ustar | gzip -c > archive.tar.gz

Answer 6

I found this somewhere else so I won't take credit, but it worked better than any of the solutions above for my mac specific issues (even though this is closed):

tar zc --exclude __MACOSX --exclude .DS_Store -f <archive> <source(s)>