How can I automate a series of tests in Perl?

Question

i, I have to run about a 100 tests on my program and doing it one-by-one is killing my time bad so I am looking to write a script. I write an individual test like so

./program test_1 > out_1

and then compare it to the output that I am looking for like so

diff -urpb expected_out_1 out_1 > diff_1

Can somebodyhelp me to write a short Perl script that runs all the tests. Note however that the names above are placeholders and the we cannot do a loop over test_*. All the tests howver have an extension X.test and the corresponding expected output files have extension X.out. (notice the names are the same)

Answer 1

my @files = glob("*.test");

foreach my $file ( @files) {
   $file =~ /^(.*?)\.test$/;
   my name = $1;
   system("program $file > $name.out");
   system("diff -urpb expected_$name.out $name.out > $name.diff");
}

Answer 2

Try this:

#!/usr/bin/perl

$TestProgram        = "./program";
$TestNames_FileName = "test.list";

# Read all Test names ++++++++++++++++++++++++++++
open(FILE, $TestNames_FileName);
while (<FILE>) {
    chomp;
    $Command1 = "$TestProgram '$_.test' > '$_.out'";
    $Command2 = "diff -urpb '$_.expected_out' '$_.out' > '$_.diff'";
    system $Command1
    system $Command2
}
close(FILE);

To run this, you need a file named 'test.list' that contain a list of test line by line without empty last line.

Hope this helps.

Answer 3

if you like bash:

for i in `ls yourprogramfiles`
do
program $i >temp
diff -urpb expected_out_1 temp > diff_1${i}
rm -f temp
done

Answer 4

You might want to consider rewriting your tests to make use of the Test Anything Protocol which has wonderful support in Perl.

use Test::More;
my @files = glob("*.test"); 
plan tests => scalar @files; 
for my $file (@files) { 
  (my $name = $file) =~ s/\.test$//;
  system("program $file > $name.out");
  is system("diff -urpb expected_$name.out $name.out > $name.diff"),
    0, $name;
}

You can run this directly -- TAP is human-readable, and the output will look like

1..100
ok 1 - test_a
ok 2 - test_b
...
ok 100 - zargle_fnargle

or you can run it inside of a test harness like prove, which will give you a nice status display while the tests run and summarize the results with output like

100....ok
All tests successful.
Files=1, Tests=100, 200 wallclock secs (blah blah blah)

or perhaps

100....
# Failed test 'tricky_test'
# at tests.t line 7

# Failed test 'another_tough_one'
# at tests.t line 7

# Looks like you failed 2 tests out of 100.
DIED. FAILED tests 3, 54
Failed 2/100 tests, 98.00% okay
(etc. etc. etc.)

It's a very useful tool. :)