Are there any good automated test suites for Perl?

Question

Can someone suggest some good automated test suite framework for Perl?

Answer 1

I'd go for Test::More, or in general, anything that outputs TAP

Answer 2

As of now, we use Test::More but current issue is that we have to run all the test files manually for testing. What I am looking for is a more of automated framework which can do incremental testing/build checks etc.

A wrapper around Test::More for that would be ideal but anything better and more functional would be fine too.

I am going through PerlUnit to see if that helps.

Answer 3

If I understand you correctly you are looking for TAP::Harness

Answer 4

If you're using ExtUtils::MakeMaker or Module::Build, then you can run all your tests automatically by entering the command "make test" or "Build test", which will execute any *.t files in your project's t/ subfolder.

If you're not using either of these, then you can use TAP::Harness to automate execution of multiple test scripts.

To actually write the tests, use Test::More or any of the modules that others have suggested here.

Answer 5

Are you aware of the 'prove' utility (from App::Prove)? You can tell it to run all the tests recursively in a given directory, with or without verbosity, etc.

Answer 6

Personally, I like Test::Most, its basically Test::More with some added cool features.

Answer 7

As long as you are using tests that produce TAP (Test Anything Protocol) output you might find this to be useful: http://sourceforge.net/projects/smolder

Answer 8

Check out CPAN Testers, which have a lot of tools for automated testing. Most of that should be on CPAN so you'll be able to modify it to meet your needs. It's also very easy to write your own tester using TAP::Harness.

What exactly do you need to do and how are you trying to fit it into your process?

Answer 9

It really depends on what you're trying to do, but here's some background for much of this.

First, you would generally write your test programs with Test::More or Test::Simple as the core testing program:

use Test::More tests => 2;

is 3, 3, 'basic equality should work';
ok !0, '... and zero should be false';

Internally, Test::Builder is called to output those test results as TAP (Test Anything Protocol). Test::Harness (a thin wrapper around TAP::Harness), reads and interprets the TAP, telling you if your tests passed or failed. The "prove" tool mentioned above is bundled with Test::Harness, so let's say that save the above in the t/ directory (the standard Perl testing directory) as "numbers.t", then you can run it with this command:

prove --verbose t/numbers.t

Or to run all tests in that directory (recursively, assuming you want to descend into subdirectories):

prove --verbose -r t/

(--verbose, of course, is optional).

As a side note, don't use TestUnit. Many people recommend it, but it was abandoned a long time ago and doesn't integrate with modern testing tools.

Answer 10

we have to run all the test files manually for testing

You certainly want to be using prove (runs your test) and/or Module::Build (builds your code and then runs your tests using the same test harness code which prove uses internally.)

Answer 11

For automated testing in perl take a look at Test::Harness, which contains the prove tool.

The prove tool can be executed with the following command:

prove -r -Ilib t

This will recursivly test all *.t files in the 't/' directory, while adding lib to the include path.

Answer 12

The test suite framework of choice is Test::Harness, which takes care of controlling a test run, collecting the results, etc.

Various modules exist to provide certain kinds of tests, the most common of which can be found in Test::Simple and Test::More (both are included in the Test-Simple distribution). The entire Test namespace on the CPAN is dedicated to specialized unit-testing modules, the majority of which are designed to be run under Test::Harness.

By convention, tests are stored in the t/ directory of a project, and each test file uses the file extension .t ; tests are commonly run via

 prove t/*.t

Module distributions typically include a make target named 'test' that runs the test suite before installation. By default, the CPAN installation process requires that tests pass after the build before a module will be installed.

Answer 13

Have you seen smolder?

"Smoke Test Aggregator used by developers and testers to upload (automated or manually) and view smoke/regression tests using the Test Anything Protocol. Details and trends are graphed and notifications provided via email or Atom feeds."

Answer 14

Hi,

You said:

"What I am looking for is a more of automated framework which can do incremental testing/build checks etc"

Still not entirely sure what you're after. As others have mentioned you want to look at things that are based on Test::Harness/TAP. The vast majority of the Perl testing community uses that framework - so you'll get much more support (and useful existing code) by using that.

Can you talk a little more about what you mean by "incremental testing/build checks"?

I'm guessing that you want to divide up your tests into groups so that you're only running certain sets of tests in certain circumstances?

There are a couple of ways to do this. The simplest would be to just use the file system - split up your test directories so you have things like:

core/
 database.t
 infrastructure.t
style/
  percritic.t
ui/
  something.t
  something-else.t

And so on... you can then use the command line "prove" tool to run them all, or only certain directories, etc.

prove has a lot of useful options that let you choose which tests are run and in which order (e.g. things like most-recently-failed order). This - all by itself - will probably get you towards what you need.

(BTW it's important to get a recent version of Test::Simple/prove/etc. from CPAN. Recent versions have much, much more functionality).

If you're of an OO mindset, or have previous experience of xUnit frameworks, than you might want to take a look at Test::Class which is a Perl xUnit framework that's build on top of the TAP/Test::Harness layer. I think it's quite a lot better than PerlUnit - but I would say that since I wrote it :-)

Check out delicious for some more info on Test::Class http://delicious.com/tag/Test::Class

If this isn't what you're after - could you go into a bit more detail on what functionality you want?

Cheers,

Adrian