When is a Test not a Unit-test?

Question

I am looking for rules like:

A test is not a unit-test if:

• it communicates with a database
• it cannot run in parallel with other tests
• uses the "environment" like registry or file system

What else is there?

Answer 1

It has no asserts!

Answer 2

See Michael Feathers' definition

A test is not a unit test if:

• It talks to the database
• It communicates across the network
• It touches the file system
• It can't run at the same time as any of your other unit tests
• You have to do special things to your environment (such as editing config files) to run it.

Answer 3

Implementing a test across multiple possibly failing units would not be a unit test.

Answer 4

In addition to whats been said, check:

This question.

Answer 5

Intricate question.

Say I am to program some business logic and all business logic needs to get to the data via some form of DAL.

Say that for the purposes of testing, I mock the DAL units (by creating "mockingbirds").

But those mockingbirds are of course, additional units in their own right. So even when using mocks, it might seem like I'm still bound to violate the idea of "no other units involved" when I want to unit-test my business logic module.

Of course, it is generally known that "creating mockingbirds for the DAL" can invalidate your very test itself on the count that your mockingbird deviates in some particular aspect from the DAL.

Conclusion : it is outright impossible to do "genuine unit-tests" on business modules that depend in any way on any kind of DAL, question mark ?

Corrolary : the only thing that can possible be ("genuinely" !) unit-tested is the DAL itself, question mark ?

Corrolary of the corrolary : given that the "DAL" is usually either an ORM or the very DML of some DBMS, and given that those products are usually bought as being "proven technology", what is the added value of doing any unit tests what so ever, question mark ?

Answer 6

A test is not a unit test if it is not testing a unit.

Seriously, that's all there is to it.

The concept of "unit" in unit testing is not well-defined, in fact, the best definition I have found so far, isn't actually a definition because it is circular: a unit in a unit test is the smallest possible thing that can be tested in isolation.

This gives you two checkpoints: is it tested in isolation? And is it the smallest possible thing?

Please note that both of these are context-dependent. What might be the smallest possible thing in one situation (say, an entire object) might in another situation just one small piece of one single method. And what counts as isolation in one situation might be in another (e.g. in a memory-managed language, you never run in isolation from the garbage collector, and most of the time that is irrelevant, but sometimes it might not be).

Answer 7

Difficult one...

For me a unit test verifies one specific piece of logic in isolation. Meaning, I take some logic, extract it from the rest (if necessary by mocking dependencies) and test just that logic - a unit (of the whole) - by exploring different kind of possible control flows.

But on the other side...can we always 100% say correct or incorrect?? Not to become philosophical, but - as also Michael says in his post:

Tests that do these things aren't bad. Often they are worth writing, and they can be written in a unit test harness. However, it is important to be able to separate them from true unit tests so that we can keep a set of tests that we can run fast whenever we make our changes.

So why shouldn't I write a unit test that verifies the logic of parsing for instance an xls file by accessing some dummy file from the file system in my test folder (like MS tests allow with the DeploymentItem)?

Of course - as mentioned - we should separate these kind of tests from the others (maybe in a separate test suite in JUnit). But I think one should also write those tests if he feels comfortable in having them there...clearly then always again remembering that a unit test should just test a piece in isolation.

What is most important in my eyes is that these tests run fast and don't take too long s.t. they can be run repeatedly and very often.