How to use automation for testing application involving highly complex calculations?

Question

Hi,

I want to following things for testing a application involving complex calculations:

1. How to use test automation tools for testing calculations?(using automation tools like QTP or open source tools)
2. How to decide coverage while testing calculations, how to design test cases?

Thanks in advance, Testmann

Answer 1

Sometimes, you have to estimate the expected conclusion, and then populate the test case from a run of the program.

It's not so much of a mortal sin, as long as you're convinced it's correct. Those tests will then let you know immediately if a code change breaks the code. Also, if you're testing a subset, it's not that big of a stretch of trust.

And for coverage? Cover every branch at least once (that is, any if or loop statements). Cover every threshold, both sides of it (for integer division that would be -1, 0, and 1 as denominators). Then add a few more in for good measure.

Answer 2

We had to test some really complex calculations in an application we built. To do this we used a tool call FitNesse, which is a wiki test harness (and open source too). It works really well when you provide it data in a table style format.

We had some code in C# that perform some VERY complex calculations. So what we did is wrote a test harness in FitNesse, and then we supplied it with a lot of test data. We worked very hard to cover all cases, so we utilized a sort of internal truth-table to ensure we were getting every possible combination of data input.

The FitNesse test harness has been invaluable to us as the complexity of the calculations has changed over time due to changing requirements. We've been able to ensure the correctness of the calculations because our FitNesse tests act as a very nice regression suite.

Answer 3

To test existing code, you should assume that the code is (mostly) correct. So you just give it some data, run it and record the result. Then use that recorded result in a test case.

When you do the next change, your output should change too and the test will fail. Compare the new result with what you'd have expected. If there is a discrepancy, then you're missing something -> write another test to figure out what is going on.

This way, you can build expertise about an unknown system.

When you ask for coverage, I assume that you can't create coverage data for the actual calculations. In this case, just make sure that all calculations are executed and feed them with several inputs. That should give you an idea how to proceed.