views:

239

answers:

2

I am looking for a test framework that suit my requirements. Following are the steps that I need to perform during automated testing:

  • SetUp (There are some input files, that needs to be read or copied into some specific folders.)
  • Execute (Run the stand alone)
  • Tear Down (Clean up to bring the system in its old state)

Apart from this I also want to have some intelligence to make sure if a .cc file changed, all the tests that can validate the changes should be run.

I am evaluating PyUnit, cppunit with scons for this. Thought of running this question to make sure I am on right direction. Can you suggest any other test framework tools? And what other requirements should be considered to select right test framework?

+3  A: 

Try googletest AKA gTest it is no worse then any other unit test framework, but can as well beat some with the ease of use. Not exactly a tool for integration testing you are looking for, but can easily be applied in most cases. This wikipedia page might also be useful for you.

Here is a copy of a sample on the gTest project page:

#include <gtest/gtest.h>

namespace {

// The fixture for testing class Foo.
class FooTest : public ::testing::Test {
 protected:
  // You can remove any or all of the following functions if its body
  // is empty.

  FooTest() {
    // You can do set-up work for each test here.
  }

  virtual ~FooTest() {
    // You can do clean-up work that doesn't throw exceptions here.
  }

  // If the constructor and destructor are not enough for setting up
  // and cleaning up each test, you can define the following methods:

  virtual void SetUp() {
    // Code here will be called immediately after the constructor (right
    // before each test).
  }

  virtual void TearDown() {
    // Code here will be called immediately after each test (right
    // before the destructor).
  }

  // Objects declared here can be used by all tests in the test case for Foo.
};

// Tests that Foo does Xyz.
TEST_F(FooTest, DoesXyz) {
  // Exercises the Xyz feature of Foo.
}

Scons could take care of building your .cc when they are changed, gTest can be used to setUp and tearDown your tests.

I can only add that we are using gTest in some cases, and a custom in-house test automation framework in almost all other. It is often a case with such tools that it might be easier to write your own than try to adjust and tweak some other to match your requirements.

One good option IMO, and it is something our test automation framework is moving towards, is using nosetests, coupled with a library of common routines (like start/stop services, get status of something, enable/disable logging in certain components etc.). This gives you a flexible system that is also fairly easy to use. And since it uses python and not C++ or something like that, more people can be busy creating test cases, including QEs, which not necessarily need to be able to write C++.

Dmitry
Execute step contains running an executable that exercises the interfaces of the component being tested. In this case TEST_F() would contain the exec or system function calling the executable followed by validation of output. Do you think gtest is good option for this case?
sand
I would recommend using scripting languages like python for this type of testing. I would choose nosetests. If you prefer to use C++ instead, sure, gtest is probably a better option then many other frameworks for the ease of use if nothing else.
Dmitry
+1  A: 

After reading this article http://gamesfromwithin.com/exploring-the-c-unit-testing-framework-jungle some time ago I went for CxxTest.

Once you have the thing set up (you need to install python for instance) it's pretty easy to write tests (I was completely new to unit tests)

I use it at work, integrated as a visual studio project in my solution. It produces a clickable output when a test fails, and the tests are built and run each time I build the solution.

f4
I recently switched to gTest, I haven't used it in a real project yet but it does seem pretty good. I like the ease of use, and not having to install python or perl like CxxTest require.
f4