ansaurus

Question

Perl: Deleting multiple re-occuring lines where a certain criteria is met

Answer 1

A:

s/^.*NULL\r?\n.*\r?\n.*\r?\n//mg;

should filter out the lines that end in NULL plus the two following lines.

Tim Pietzcker 2010-04-21 19:40:04

Hi Tim,I have tested it out by incorporating it into my script. It doesn't remove the NULL values. They are still there in the output. Please modify.Thanks for the effort.GL.

2010-04-21 20:09:58

@george-lule: "Please modify?" You have to be joking! What a great way to ask for help.perldoc perlreand figure it out. Otherwise, I suggest you try APL. Or ADA. Or Pascal. Or Brainfuck.

xcramps 2010-04-21 21:32:03

Changing the `/mg` to `/sg` should do the trick

Zaid 2010-04-22 06:09:38

@Zaid: `/m` is intentional - I want `^` to match the start of the line (instead of start of string); I *don't* want the dot to match newlines (which is what `/s` would do).@george-lule: This regex action has to be applied to the entire input, not single lines. It works on the test data you provided.

Tim Pietzcker 2010-04-22 06:35:27

@Tim: My bad, I should've paid attention to the `\n` s in your regex

Zaid 2010-04-22 09:33:15

Answer 2

+1 A:

Looks like a good candidate for a little input record separator ($/) trickery. The idea is to manipulate it so that it deals with one record at a time, rather than the default single line.

use strict;
use warnings;

$^I = '.bak';

open my $dataIn, '<', 'george_perl.txt' or die "Can't open data file: $!";
open my $dataOut, '>', 'gen_results.txt' or die "Can't open output file: $!";

{
    local $/ = "\n\t"; # Records have leading tabs

    while ( my $record = <$dataIn> ) {

        # Skip header & records that contain 'NULL'
        next if $record =~ /NULL|Event_time/;

        # Strip out the unwanted yik-yak
        $record =~ s/SubNetwork.*BtsSiteMgr=//s;

        # Print record to output file
        print $dataOut $record;
    }
}

close $dataIn;
close $dataOut;

Pay attention to the following:

use of the safer three-argument form of open (the two-argument form is what you've shown)
use of scalar variables rather than barewords for defining filehandles
use of the local keyword and extra curlies to modify the definition of $/ only when needed.
the second s in s/SubNetwork.*BtsSitMgr=//s allows matches over multiple lines as well.

Zaid 2010-04-21 19:41:01

Hi Zaid,Your suggestion clears only the line containing the NULL value but leaves the rest of the entry associated with it(the two lines below). These 2 lines must also go. Like mentioned, for cease time NULL, that entire record including the element must go. Kindly modify.Thanks for the effortGL.

2010-04-21 19:58:23

@george-lule: Are those tabs or 3 spaces at the beginning of each record? If you could explain how your records are separated, I can tell you what the `$/` string should be.

Zaid 2010-04-21 20:00:39

Hi Zaid,They are tabs.RegardsGL.

2010-04-21 21:06:40

@george-lule: Updated. Changed the `"\n "` to `"\n\t"`

Zaid 2010-04-22 05:46:37

Thanks Zaid, but the issue has been solved by FM above. Let me try out your code though, never know when it can come in handy ;-)

2010-04-22 05:52:34

@george-lule: I just tested out my code and updated it accordingly. It should work just as you want it to.

Zaid 2010-04-22 05:59:12

Answer 3

+1 A:

Your data arrives in sets of 3 lines, so one approach is to organize the parsing that way:

use strict;
use warnings;

# Ignore header junk.    
while (<>){
    last unless /\S/;
}

until (eof) {
    # Read in a set of 3 lines.
    my @lines;
    push @lines, scalar <> for 1 .. 3;

    # Filter and clean.
    next if $lines[0] =~ /\sNULL\s/;
    $lines[2] =~ s/.+BtsSiteMgr=//;

    print @lines[0,2];
}

FM 2010-04-21 20:46:31

Hi FM,I am getting a blank output, after integrating your suggestion, code is below. Where could I be going wrong?#!/usr/bin/perluse strict;use warnings;open (DATAIN,"<george_perl.txt")|| die("can't open datafile: $!"); open (DATAOUT,">gen_results.txt")|| die("can't open datafile: $!"); while (<DATAIN>){ last unless /\S/;}until (eof) { my @lines; push @lines, scalar <> for 1 .. 3; next if $lines[0] =~ /\sNULL\s/; $lines[1] = ''; $lines[2] =~ s/.+BtsSiteMgr=//; print DATAOUT @lines;}close DATAIN;close DATAOUT;

2010-04-21 21:16:30

@george-lule You are missing your `DATAIN` file handle in two places: `until (eof DATAIN)` and `... scalar <DATAIN> ...`.

FM 2010-04-21 21:51:14

Thanks FM.The header was still a problem but everything is now working. Thanks alot once again.

2010-04-22 05:46:02

ansaurus

tags:

views:

answers:

Perl: Deleting multiple re-occuring lines where a certain criteria is met

related questions