Perl: Extract data from logs into a report

Question

I have some logs in a directory: /test/report. The logs are named: a.log, b.log, c.log.

The content of a.log is below:

Input 1
----
Number of records := 101


Input 2
---
Num of of records :=  101


Input 3
---
Num of records := 101

The content of b.log is below:

Input1
-------
Number of records := 88 

Input 2
-----
Num of of records :=  88 

Input 3
---
Num of records := 86

The content of c.log is below:

Input1
-------
Number of records := 4 

Input 2
-----
Num of of records :=  3 

Input 3
---
Num of records := 4

My question is, how can I use Perl to open the files and then extract the number out so that I can have generate a report that looks like below:

       Input 1          Input 2           Input 3        Result
A          101              101               101            OK
B           88               88                86         Error
C            4                3                 4         Error

Answer 1

• use glob function to find all the applicable log files

• iterate over all the found log files using a while loop

• open each file using open function (don't forget to use 3-argument form of open and do error handling).

• Go through all the lines in the file by either using File::Slurp or the goold old while(<$filehandle>) loop.

• Parse the contents of lines using regular expressions.

• You might need to implement very simple state machine depending on the exact structure of the log file which you didn't describe in precise detail

• when your regex matches a value of interest (e.g. # of records of # of input), capture it using parenthesis in the regex and then after the match save using $1, $2 etc... variables where the matched pieces are saved.

• If needed, aggregate the data, probably by saving in a hash (keyed by input file name/type) and the value being the hashref of inputs keyed by input type (if input types are guaranteed to be small #s, you can use array ref instead)

• Iterate over your data structure and print row by row.

Use Perldoc or any of the Perl books for reference to any of the above steps/concepts/functions; and feel free to ask questions on SO if you run into specific trouble withe any of the specific code.

Answer 2

Here's the extraction portion, and that's most of the problem. DVK outlined the other steps.

This looks for .log files in the current working directory:

my %hash;

foreach my $file ( glob( "*.log" ) ) {
    my $contents = do { local( @ARGV, $/ ) = $file; <> };

    print $contents;
    my %results = $contents =~ m/Input\s*(\d+).*?Num.*?records\s+:=\s+(\d+)/sg;
    $hash{$file} = \%results;
    }

print Dumper( \%hash ); use Data::Dumper;

I might explain what I did, but then I thought you'd enjoy figuring it out on your own so you get some Perl practice. :)