bash: filter away consecutive lines from text file

Question

I want to delete from many files each instance of a paragraph. I call paragraph a sequence of lines.

For example:

my first line
my second line
my third line
the fourth
5th and last

the problem is that I only want to delete them when they appear as a group. For example, if

my first line

appears alone I don't want to delete it.

Answer 1

If you are able to use Perl, you can do it in one line like this:

perl -0777 -pe 's/my first line\nmy second line\nmy third line\nthe fourth\n5th and last\n//g' paragraph_file

the explanation is in perlrun:

The special value 00 will cause Perl to slurp files in paragraph mode. The value 0777 will cause Perl to slurp files whole because there is no legal byte with that value.

Sample input:

my first line
my second line
my third line
the fourth
5th and last
hey
my first line
my second line
my third line
the fourth
5th and last

hello
my first line

Output:

$ perl -0777 -pe 's/my first line\nmy second line\nmy third line
\nthe fourth\n5th and last\n//g' paragraph_file
hey

hello
my first line

Answer 2

You can do it with sed:

sed '$!N; /^\(.*\)\n\1$/!P; D' file_to_filter

Answer 3

@OP, i see you accepted the answer whereby your paragraph sentences are "hardcorded", so i assume those paragraphs are always the same? its that's true, you can use grep. Store the paragraph you want to get rid of in a file eg "filter", then use -f and -v option of grep to do the job,

grep -v -f filter file