perl help to parse log file based on time input

Question

Perl newbie here. I have a log file that I need to parse out "Backup succeeded" and any "Error:" entries. I tried parsing the log file by using unix cat and piping it to grep. I got the information that I want, but I would like try this in perl and to also have the option to pass a date parameter and give me the lines based on the date I need.

Sample of log file output: (Backup succeeded)

Wed Jun 09 06:14:25 2010: db2.cal.mil.mad:backup:INFO: flush-logs-time=00:00:00
Wed Jun 09 06:14:25 2010: db2.cal.mil.mad:backup:INFO: backup-time=06:14:23
Wed Jun 09 06:14:25 2010: db2.cal.mil.mad:backup:INFO: backup-status=Backup succeeded
Wed Jun 09 06:14:25 2010: db2.cal.mil.mad:backup:INFO: Backup succeeded

Sample of log file output: (Error:)

Wed Jun 09 05:00:03 2010: rip1.mil.mad:backup:ERROR: mysql-zrm appears to be already running for this backupset
Wed Jun 09 05:00:03 2010: rip1.mil.mad:backup:ERROR: If you are sure mysql-zrm is not running, please remove the file /etc/mysql-zrm/rip1.mail.mad/.mysql-zrm.pid and restart mysql-zrm

**I would like a text and/or email with this information. Like so, but with the option to pass in the date I need.

Wed Jun 09 05:00:03 2010: rip1.mil.mad:backup:ERROR: mysql-zrm appears to be already running for this backupset
Wed Jun 09 05:00:03 2010: rip1.mil.mad:backup:ERROR: If you are sure mysql-zrm is not running, please remove the file /etc/mysql-zrm/rip1.mail.mad/.mysql-zrm.pid and restart mysql-zrm
Wed Jun 09 06:14:25 2010: db2.cal.mil.mad:backup:INFO: backup-status=Backup succeeded

If you would please provide me with some perl code and/or ideas to get started. I would appreciate the help. Thank you.

Answer 1

Here's a simple script. The file name to scan and the target date are hard-coded. Matches are printed to STDOUT.

BTW, this code is totally untested. I typed it into the text box in my browser.

use strict;
use warnings;

my $logpath = './bar/log';
my $target = 'Jun 09 2010';

open my $fh, '<', $logpath or die "Error opening $logpath $!\n";

while (my $line = <$fh> ) { 

    next unless date_match( $target, $line );

    next unless my $result = got_error($line) // got_backup($line);    

    print $result;
}

sub got_backup {
     my $line = shift;

     return unless $line =~ /backup-status=Backup succeeded/;

     return $line;
}

sub got_error {
     my $line = shift;

     return unless $line =~ /:ERROR:/;

     return $line;
}


# Take a line and a target date.  Compare the date derived from the line to
# the target, and returns true if they match.
# Also always returns true if target is not defined

sub date_match {
    my $target = shift;
    my $line = shift;

    return 1 unless defined $target;  # Always true if target is undefined.

    # Where did that god-awful date format come from?  Yech.
    my $date = extract_date($line);

    return $date eq $target;
}

# Simple extract of date using split and join with extra variables
# to make it newbie friendly.
# IMO, it would be a good idea to switch to using DateTime objects and 
#    DateTime::Format::Strptime

sub extract_date {
    my $line = shift;

    my @parts = split /:/, $line;
    my $date = join ':' @parts[0..2];
    @parts = split /\s+/, $date;

    $date = @parts[1,2,4]; 

    return $date;        
}

• You can use Getopt::Long to get a filename and target date.

• It would be a good idea to use a more robust date/time parsing and comparison scheme. DateTime and friends are very good, powerful modules for date manipulation. Check them out.

• If you are processing tons of data and need to be more efficient, you can avoid copying $line everywhere in a number of ways.

• For future reference, if you post a little code, you'll get better responses

Answer 2

#!/usr/bin/perl

# usage example: <this script> Jun 09 2010 <logfile>

use strict;
use warnings;

my ($mon,$day,$year) = ($ARGV[0],$ARGV[1],$ARGV[2]);

open(FH,"< $ARGV[3]") or die "can't open log file $ARGV[3]: $!\n";

while (my $line = <FH>) {
    if ($line =~ /.* $mon $day \d{2}:\d{2}:\d{2} $year:.*(ERROR:|Backup succeeded)/) {
    print $line;
}
}