ansaurus

Question

Is there a truly universal wildcard in Grep?

Answer 1

+3 A:

You could do it with Perl:

$ perl -ne 'print if /HEADER TEXT/ .. /FOOTER TEXT/' file.html

To print only the text between the delimiters, use

$ perl -000 -lne 'print $1 while /HEADER TEXT(.+?)FOOTER TEXT/sg' file.html

The /s switch makes the regular expression matcher treat the entire string as a single line, which means dot matches newlines, and /g means match as many times as possible.

The examples above assume you're cranking on HTML files on the local disk. If you need to fetch them first, use get from LWP::Simple:

$ perl -MLWP::Simple -le '$_ = get "http://stackoverflow.com";
                          print $1 while m!<head>(.+?)</head>!sg'

Please note that parsing HTML with regular expressions as above does not work in the general case! If you're working on a quick-and-dirty scanner, fine, but for an application that needs to be more robust, use a real parser.

Greg Bacon 2009-12-13 19:09:32

Answer 2

+1 A:

As pointed elsewhere, grep will work for single line stuff.

For multiple-lines (in ruby with Regexp::MULTILINE, or in python, awk, sed, whatever), "\s" should also capture line breaks, so

HEADER TEXT(.*\s*)FOOTER TEXT

might work ...

phtrivier 2009-12-13 19:09:53

You'd have to be reading the file in a mode that scans multiple lines into memory for that to work.

Jonathan Leffler 2009-12-13 19:35:42

Thanks, I added how you would do that in Ruby. IIRC, that's /g in perlish, isn't it ?

phtrivier 2009-12-13 19:54:23

Answer 3

+2 A:

The man page of grep says:

grep, egrep, fgrep, rgrep - print lines matching a pattern

grep is not made for matching more than a single line. You should try to solve this task with perl or awk.

tangens 2009-12-13 19:11:55

Answer 4

+2 A:

By definition, grep looks for lines which match; it reads a line, sees whether it matches, and prints the line.

One possible way to do what you want is with sed:

sed -n '/HEADER TEXT/,/FOOTER TEXT/p' "$@"

This prints from the first line that matches 'HEADER TEXT' to the first line that matches 'FOOTER TEXT', and then iterates; the '-n' stops the default 'print each line' operation. This won't work well if the header and footer text appear on the same line.

To do what you want, I'd probably use perl (but you could use Python if you prefer). I'd consider slurping the whole file, and then use a suitably qualified regex to find the matching portions of the file. However, the Perl one-liner given by '@gbacon' is an almost exact transliteration into Perl of the 'sed' script above and is neater than slurping.

Jonathan Leffler 2009-12-13 19:12:18

Answer 5

+3 A:

When I need to match several characters, including line breaks, I do:

[\s\S]*?

Note I'm using a non-greedy pattern

Rubens Farias 2009-12-13 19:16:20

Thanks guys! What a friendly, useful site. I forgot to mention that I was using grep search in BBEdit, this works wonderfully. You all rock!

Tom B 2009-12-13 19:39:18

Answer 6

A:

here's one way to do it with gawk, if you have it

awk -vRS="FOOTER" '/HEADER/{gsub(/.*HEADER/,"");print}' file

2009-12-14 00:02:32

ansaurus

tags:

views:

answers:

Is there a truly universal wildcard in Grep?

related questions