How to test function call order

Question

Considering such code:

class ToBeTested {
public:
  void doForEach() {
    for (vector<Contained>::iterator it = m_contained.begin(); it != m_contained.end(); it++) {
       doOnce(*it);
       doTwice(*it);
       doTwice(*it);
    }
  }
  void doOnce(Contained & c) {
    // do something
  }
  void doTwice(Contained & c) {
    // do something
  }

  // other methods
private:
  vector<Contained> m_contained;
}

I want to test that if I fill vector with 3 values my functions will be called in proper order and quantity. For example my test can look something like this:

tobeTested.AddContained(one);
tobeTested.AddContained(two);
tobeTested.AddContained(three);

BEGIN_PROC_TEST()
SHOULD_BE_CALLED(doOnce, 1)
SHOULD_BE_CALLED(doTwice, 2)
SHOULD_BE_CALLED(doOnce, 1)
SHOULD_BE_CALLED(doTwice, 2)
SHOULD_BE_CALLED(doOnce, 1)
SHOULD_BE_CALLED(doTwice, 2)

tobeTested.doForEach()
END_PROC_TEST()

How do you recommend to test this? Are there any means to do this with CppUnit or GoogleTest frameworks? Maybe some other unit test framework allow to perform such tests?

I understand that probably this is impossible without calling any debug functions from these functions, but at least can it be done automatically in some test framework. I don't like to scan trace logs and check their correctness.

UPD: I'm trying to check not only the state of an objects, but also the execution order to avoid performance issues on the earliest possible stage (and in general I want to know that my code is executed exactly as I expected).

Answer 1

Some mocking frameworks allow you to set up ordered expectations, which lets you say exactly which function calls you expect in a certain order. For example, RhinoMocks for C# allows this.

I am not a C++ coder so I'm not aware of what's available for C++, but that's one type of tool that might allow what you're trying to do.

Answer 2

You could check out mockpp.

Answer 3

You should be able to use any good mocking framework to verify that calls to a collaborating object are done in a specific order.

However, you don't generally test that one method makes some calls to other methods on the same class... why would you?

Generally, when you're testing a class, you only care about testing its publicly visible state. If you test anything else, your tests will prevent you from refactoring later.

I could provide more help, but I don't think your example is consistent (Where is the implementation for the AddContained method?).

Answer 4

http://msdn.microsoft.com/en-au/magazine/cc301356.aspx

This is a good article about Context Bound Objects. It contains some so advanced stuff, but if you are not lazy and really want to understand this kind of things it will be really helpful.

At the end you will be able to write something like: [CallTracingAttribute()] public class TraceMe : ContextBoundObject {...}

Answer 5

Instead of trying to figure out how many functions were called, and in what order, find a set of inputs that can only produce an expected output if you call things in the right order.

Answer 6

You could use ACE (or similar) debug frameworks, and in your test, configure the debug object to stream to a file. Then you just need to check the file.

Answer 7

If you're interested in performance, I recommend that you write a test that measures performance.

Check the current time, run the method you're concerned about, then check the time again. Assert that the total time taken is less than some value.

The problem with check that methods are called in a certain order is that your code is going to have to change, and you don't want to have to update your tests when that happens. You should focus on testing the actual requirement instead of testing the implementation detail that meets that requirement.

That said, if you really want to test that your methods are called in a certain order, you'll need to do the following:

1. Move them to another class, call it Collaborator
2. Add an instance of this other class to the ToBeTested class
3. Use a mocking framework to set the instance variable on ToBeTested to be a mock of the Collborator class
4. Call the method under test
5. Use your mocking framework to assert that the methods were called on your mock in the correct order.

I'm not a native cpp speaker so I can't comment on which mocking framework you should use, but I see some other commenters have added their suggestions on this front.