What does unit testing mean to you?

Question

G'day,

I am working with a group of offshore developers who have been using the term unit testing quite loosely.

Their QA document talks about writing unit tests and then performing unit testing of the system.

This doesn't line up with my interpretation of what unit testing is at all.

I am used to unit testing being a test or suite of tests that are being used to exercise a single class, usually as a black box. The class under test may require other classes to be included by the implementation but generally it is a single class that is being exercised by the unit test(s).

Then you have system functional testing, intergration testing, acceptance testing, etc.

I want to know is this a bit pedantic on my part? Or is this what you think of when referring to unit tests and unit testing?

Edit: Rob Wells. I need to clarify that approaching such testing from a black box perspective is only one aspect. When using mock objects to verify internal behaviours, you are really testing from a white box perspective because you know what you want to happen inside the box.

Answer 1

Unit tests are generally used by developers to test isolated sections of code. They cover border cases, error cases, and normal cases. They are intended to demonstrate the correctness of a limited segment of code. If all of your unit tests pass, then you have demonstrated that your isolated segments of code do what they are supposed to do.

When you do integration testing, you are looking at end-to-end cases, to see if all the segments that have passed unit testing work together. Functional testing checks to see if the code meets the requirements as specified. Acceptance testing is done by end users to see if they approve of the final product.

Answer 2

I have heard of techniques in which many of the unit tests are done first, and development is done around them. Someone has just commented saying that this is "Test Driven Development" - TDD (Thanks Elie).

But if this is an offshore operation that's possibly going to charge you more money because they're spending time doing these unit tests - then I'd be careful. Get a second opinion from someone, experienced with unit tests who will verify that they're actually doing as they say.

From my understanding, unit testing will add a bit more time to any development project, but of course may offer some quality control. Nonetheless, this is the type of quality control I would want with an in house project. This may just be something the offshore company throws out there to give you a warm fuzzy.

Answer 3

Wikipedia would seem to suggest that unit testing is about testing the smallest amount of code which would be a method on a class in the case of OO programming.

Some may have a more general term of what they mean by unit tests, where some may think of some integration tests as being unit tests where the unit is a mixture of components.

Answer 4

This is almost a repeat of the "What is a 'Unit'?" question.

"Unit" can be defined flexibly. If their document doesn't have a definition of "unit", you'll need to clarify that.

It might be that they think of unit as a big assembly of code. Which is not the most desirable definition.

While I agree that you have several layers of testing (unit, module, package, application), I also think that much of this can be done with unit testing tools. Leading to "what is a unit?" questions coming up all the time.

Unit depends on context. For an individual developer, unit must be Class. Sometimes, it will also mean module or package.

For a team, however, their unit may be a package or a complete application.

Answer 5

What does it matter what we think? The issue here is your unhappiness with the terms they use in the document. Why don't you discuss it with them?

Answer 6

There is no reason why unit tests can't span multiple classes, or even submodules, as long as the test is treating only one consistent business operation. Think about "calculateWage", a method of a BO that uses different strategies to calculate the salary of a person. That's one unit test in my opinion.

Answer 7

There is a difference between the process you use to test and the technology that is used to support it. The various frameworks used for unit testing are generally very flexible and can be used for testing small units of code, large units and even testing entire processes. This flexibility can lead to confusion.

My recommendation is that whatever specific methodology or process you adopt that you segregate the various Unit Test into distinct assemblies or modules. The exact arrangement depends on your code and your company's organization.

The accumulative effect of using the Unit Test framework is that much of the testing of the code is automated. Adopted correctly developers can evaluate changes to the code code better with out going through a full Q&A process. As for the Q & A Process itself it makes their time more productive as the quality of the code coming out of development should be higher.

Understand it is not THE answer to all quality issues it just a useful tool like the other you use.

Answer 8

there is a traditional view of http://en.wikipedia.org/wiki/Software_testing as part of http://en.wikipedia.org/wiki/Software_engineering. but i like the idea of an agile unit test: a test is an agile unit test if it is fast enough so that the programmers always run it.

Answer 9

Ten years ago, before the current usage of "unit testing" as tests written in code, the same designation was applied to manual tests. I worked for a software development firm with a very formalized software development process. We had to write "unit tests" before writing any code. In that era, the unit tests were written in a text document (such as in Word). They described the exact steps that the user was to follow in using the app. For example, they described the exact input to type on the page to set up a new customer. Or, the user was to search for a particular product, and see that the displayed information matched the test document. So, the test was a script that the tester followed, where they also recorded the results. When the new incarnation of unit testing came along, it was confusing for a while trying to figure out if they meant the old, human tests or the new, coded tests.

Answer 10

A unit test is the smallest and only piece of confidence you can get yourself on your way to being done. That is what matters, iteratively building a shield against regression and spec deviation, not how you actually integrate it to your Object-Oriented architecture.

Answer 11

I try to implement unit tests to test only a single method. and I make an effort to crete "mock" classes for dependant classes and methods used by the method I am testing... ... so that the exercise of the code in that method does not in fact call code in other methods the unit test is not supposed to be "Testing" (There are other unit tests for those methods) This way, a failure of the unit test reliably indicates a failure of the method the unit test is testing...

Mock classes are designed to "simulate" the interface and behavior of dependant classes so that the method I am testing can call them and they will behave ina standard, well-defined way according to system requirements. In order to make this approach work, calls to such dependant classes and to their methods must be made on a well defined interface, so that the "tester" process can "inject" the Mock version of teh dependant class into the class being tested instead of the actual production version... . This is kinda like a common design pattern referred to as "Dependency Injection", or "Inversion of Control" (IOC)

There are several third party tools on the market to help you implement this kind of pattern. One I have heard of is called "Rhino-Mock" or something like that...

Edit: Rob Wells. @Charles. Thanks for this. I'd forgotten using mock objects to completely replace using other classes except for the one under test.

A couple of other things I've remembered after you mentioning mock objects is that:

• they can be used to simulate errors being returned by the included classes.
• they can be used to raise specific exceptions to check exception handling in the class under test.
• they can be used to simulate items where setup costs are high, e.g. a large SQL DB back end.
• they can be used to verify the contents of an incoming request.

For more information, have a look at Martin Fowler's paper called "Mocks Aren't Stubs" and The Pragmatic Programmers's article "Mock Objects"

Answer 12

I lead a group of offshore team too. Supposely we have a set of unit tests...but it doesn't mean much. :) So we rely much more on the functional and testers for quality. The inherit problem with unit testing is that you have perfect knowledge of the functionals, and you trust the developers. In the real world, that's hard to assume..