regexp in bash (downloading output form regexp)

Question

I have got a file that look like:

<a href="some-adress">some-txt</a>
<a href="some-adress">some-txt</a>
<a href="some-adress">some-txt</a>
...

I need to download all files that are as "some-adress", how can I do that using only bash?

Answer 1

Why don't you use wget ? It already have that feature :

wget -i --force-html yourfile.html

Answer 2

cut -f 2 -d '"' file-with-addresses.txt

cut is included in all posix shells. This command will split the line using the " as the delimiter and return the second "field". To download using wget Adam Rosenfield's method is fine.

cut -f 2 -d '"' file-with-addresses.txt | xargs wget

Answer 3

Here's one way to do that using a combination of sed, xargs, and wget:

sed -n 's/.*<a href="\([^"]*\)">.*/\1/p' input-file | xargs wget