What is a good regression testing framwork for software applications?

Question

Am looking for a regression test framework where I can add tests to.. Tests could be any sort of binaries that poke an application..

Answer 1

You did not indicate what language you are working in, but the xUnit family is available for a lot of different languages.

/Allan

Answer 2

It also depends heavily what kind of application you're working on. For a commandline app, for example, its probably easy enough to just create a shell script that calls it with a whole bunch of different options and compares its result to a previously known stable version, warning you if any of the output differs so that you can check whether the change is intentional or not.

If you want something more fancy, of course, you'll probably want some sort of dedicated testing framework.

Answer 3

I assume you are regression-testing a web application?

There are some tools in this kb article from Microsoft

And if I remember correctly, certain editions of Visual Studio also offer its own flavor of regression testing tools as well.

But if you just want a unit testing framework, the xUnit family does it pretty well.

Here's JUnit and NUnit.

Answer 4

This really depends on what you're trying to do, but one of the features of the new Test::Harness (disclaimer: I'm the original author and still a core developer) is that if your tests output TAP (the Test Anything Protocol), you can use Test::Harness to run test suites written in multiple languages. As a result, you don't have to worry about getting "locked in" to a particular language because that's all your testing software supports. In one of my talks on the subject, I even give an example of a test suite written in Perl, C, Ruby, and HTML (yes, HTML -- you'd have to see it).

Answer 5

Just thought I would tell you guys what I ended up using..

QMtest ::=> http://www.qmtest.com

I found QMTest to full fill my needs. Its extensible framework, allows you to write very flexible test classes. Then, these test classes could be instantiated to large test suites to do regression testing.

QMTest is also very forward thinking, it allows for weak test dependencies and the creation of test resources. After a while of using QMTest, I started writing better quality tests. However, like any other piece of complex software, it requires some time to learn and understand the concepts, the API is documented and the User Manual give a good introduction. With sometime in your hand, I think QMTest is well worth it.