I am interested in the following scenario specifically. Suppose you have team that writes production code and a team that writes automatic tests. The team that writes automatic tests has a dedicated framework intended to write the automatic tests. Should the testing team write unit tests for their framework although the framework is not used in production?
+1
A:
Yes, if only to test that the framework generates sufficient test coverage.
Chris Jester-Young
2010-04-07 08:20:30
+4
A:
Hell yes!
TDD gives you an edge in developing, it's not just to please the customer. It allows you to write testable, reusable and modular code. I would unit test everything that has to work, expecially if you expect to change it often (refactor to add new features).
Manrico Corazzi
2010-04-07 08:22:28
+2
A:
The testing team should do whatever that can increase their trust in the results delivered by their framework.
This includes testing, code reviews, quality standards, ...
mouviciel
2010-04-07 08:25:31
+4
A:
I've been in that situation and what I did is use the test suite for the production code also as a test suite for the testing framework. Presumably, all the features of the framework were actually used, so if tests failed without a change in production code, then there must be a problem in the testing framework.
It worked OK-ish - running those tests took much longer than having a dedicated test-testsuite would have, and sometimes I wouldn't run all of them and have a problem turn up on the production build server. Diagnosing such problems took much longer than it would have with a test-testsuite.
All in all, I never felt comfortable with it and would really recommend having dedicated tests for the test framework as well. From the point of view of the test-writing team, the testing framework is production code. And if the testing framework ever gets used by anyone else, whose test suites you don't have access to...
Michael Borgwardt
2010-04-07 08:27:18
A:
This question reminds me of a story: Once upon a time there was a company. This company believes in automatic tests. This belief is so strong it leads to creating a group whose sole purpose is writing these automatic tests. All the most ardent believers are allowed to join this group. There was much rejoicing!
Then one day, it's discovered that this automatic testing group, despite its mission, doesn't use automatic tests for its own work. Stones are cast, yada yada.
I'm just saying... I would think that any testing framework would have pretty solid test coverage.
ndp
2010-04-11 04:57:29
You can look on my question this way. But I think it is naive to decide to test non production code based on this argument. Maintaining tests costs money at the end and personally I think that you should test only the automation framework and not every bit of the automation lab. On the other hand, in production code your aim is to test as much as you can. This is the reason for my question.
Ikaso
2010-04-11 07:35:04
I am being flip, but my point is that the same evaluation criteria can be applied. Why do we write automated tests even they they might add 15% to the cost of development? In the end, the answer is _because it increases our confidence in the code_. It's not black and white-- the question is should we write 1 more test? If we already have confidence in the code, or having confidence is not that important, there's no need to write the test. I would apply that guideline whether the consumer of the software is a commercial customer, fellow developer or my little brother.
ndp
2010-04-11 15:23:34