How do I recursively ftp only certain file types from a linux server using the command line?

Question

I want to download only .htm or .html files from my server. I'm trying to use ncftpget and even wget but only with limited success.

with ncftpget I can download the whole tree structure no problem but can't seem to specify which files I want, it's either all or nothing.

If I specify the file type like this, it only looks in the top folder:

ncftpget -R -u myuser -p mypass ftp://ftp.myserver.com/public_html/*.htm ./local_folder

If I do this, it downloads the whole site and not just .htm files:

ncftpget -R -u myuser -p mypass ftp://ftp.myserver.com/public_html/ ./local_folder *.htm

Can I use ncftp to do this, or is there another tool I should be using?

Answer 1

Does ncftpget understand dir globs?

Try

ncftpget -R -u myuser -p mypass ftp://ftp.myserver.com/public_html/**/*.htm ./local_folder

** means any number of directories.

Answer 2

You can do it with wget

wget -r -np -A *.htm* ftp://site/dir

Answer 3

The wget command understands standing unix file globbing syntax.

wget -r -np --ftp-user=username --ftp-password=password "ftp://example.com/path/to/dir/*.htm"

Conversely, you can use the -A option, which accepts a comma-separated list of file name suffixes or patterns to accept.

wget -A '*.htm'

The -R option is the opposite of -A, so you can use it to specify patterns NOT to fetch.

Caveat: Make sure to quote patterns! Otherwise, your shell may expand the glob itself, leading to unexpected results.

Also! See the "Using wget to recursively download whole FTP directories" question on Server Fault.