Why using Integration tests instead of unit tests is a bad idea?

Question

Let me start from definition:

Unit Test is a software verification and validation method in which a programmer tests if individual units of source code are fit for use

Integration testing is the activity of software testing in which individual software modules are combined and tested as a group.

Although they serve different purposes very often these terms are mixed up. Developers refer to automated integration tests as unit tests. Also some argue which one is better which seems to me as a wrong question at all.

I would like to ask development community to share their opinions on why automated integration tests cannot replace classic unit tests.

Here are my own observations:

1. Integration tests can not be used with TDD approach
2. Integration tests are slow and can not be executed very often
3. In most cases integration tests do not indicate the source of the problem
4. it's more difficult to create test environment with integration tests
5. it's more difficult to ensure high coverage (e.g. simulating special cases, unexpected failures etc)
6. Integration tests can not be used with Interaction based testing
7. Integration tests move moment of discovering defect further (from paxdiablo)

EDIT: Just to clarify once again: the question is not about whether to use integration or unit testing and not about which one is more useful. Basically I want to collect arguments to the development teams which write ONLY integration tests and consider them as unit tests. Any test which involve components from different layers is considered as integration test. This is to compare to unit test where isolation is the main goal.

Thank you, Andrey

Answer 1

Integration testing generally happens after unit testing. I'm not sure what value there is in testing interactions between units that have not themselves been tested.

There's no sense in testing how the gears of a machine turn together if the gears might be broken.

Answer 2

There have been studies (which I'm too lazy to go looking for at the moment) that show that the cost of fixing a bug becomes higher as you move away from the point where the bug was introduced.

For example, how much does it cost for you to fix a bug in software you haven't even pushed up to source control yet? It's your time and not much of it, I'd warrant (assuming you're any good at your job).

Contrast that with how much it costs to fix when the customer (or all your customers) find that problem. Many level of people get involved and new software has to be built in a hurry and pushed out to the field.

That's the extreme comparison. But even the difference between unit and integration tests can be apparent. If your code fails in integration, you start holding up all your team-mates, testers and so on, rather than just yourself.

We wouldn't dream of replacing our unit tests with integration tests since:

1. Our unit tests are automated as well.
2. They form the beginning of the integration tests. All unit tests are rerun in the integration phase to check that the integration itself hasn't broken anything, and then there are the extra tests that have been added by the integration team.

Answer 3

• Integration tests are slow.
• Integration tests may break different reasons (it is not focused and isolated). Therefore you need more debugging on failures.
• Combination of scenarios are to big for integration test when it is not unit tested.

Mostly I do unit tests and 10 times less integration tests (configuration, queries).

Answer 4

I think coverage is the main issue.

A unit test of a specific small component such as a method or at most a class is supposed to test that component in every legal scenario (of course, one abstracts equivalence classes but every major one should be covered). As a result, a change that breaks the established specification should be caught at this point.

In most cases, an integration uses only a subset of the possible scenarios for each subunit, so it is possible for malfunctioning units to still produce a program that initially integrates well.

It is typically difficult to achieve maximal coverage on the integration testing for all the reasons you specified below. Without unit tests, it is more likely that a change to a unit that essentially operates it in a new scenario would not be caught and might be missed in the integration testing. Even if it is not missed, pinpointing the problem may be extremely difficult.

I am not sure that most developers refer to unit tests as integration tests. My impression is that most developers understand the differences, which does not mean they practice either.

Answer 5

• Unit tests focus on testing an individual component and do not rely on external dependencies. They are commonly used with mocks or stubs.
• Integration tests involve multiple components and may rely on external dependencies.

I think both are valuable and neither one can replace the other in the job they do. I do see a lot of integration tests masquerading as unit tests though having dependencies and taking a long time to run. They should function separately and as part of a continuous integration system.

Integration tests do often find things that unit tests do not though...

Answer 6

In many cases you need both. Your observations are right on track as far as I'm concerned with respect to using integration tests as unit tests, but they don't mean that integration tests are not valuable or needed, just that they serve a different purpose. One could equally argue that unit tests can't replace integration tests, precisely because they remove the dependencies between objects and they don't exercise the real environment. Both are correct.

Answer 7

Integration tests let you check that whole use cases of your application work.

Unit tests check that low-level logic in your application is correct.

Integration tests are more useful for managers to feel safer about the state of the project (but useful for developers too!).

Unit tests are more useful for developers writing and changing application logic.

And of course, use them both to achieve best results.

Answer 8

Integration tests tell you whether it's working. Unit tests tell you what isn't working. So long as everything is working, you "don't need" the unit tests - but once something is wrong, it's very nice to have the unit test point you directly to the problem. As you say, they serve different purposes; it's good to have both.

To directly address your subject: integration tests aren't a problem, aren't the problem. Using them instead of unit tests is.

Answer 9

It's all about reducing the iteration time.

With unit tests, you can write a line of code and verify it in a minute or so. With integration tests, it usually takes significantly longer (and the cost increases as the project grows).

Both are clearly useful, as both will detect issues that the other fails to detect.

OTOH, from a "pure" TDD approach, unit tests aren't tests, they're specifications of functionality. Integration tests, OTOH, really do "test" in the more traditional sense of the word.

Answer 10

I find integration tests markedly superior to unit tests. If I unit test my code, I'm only testing what it does versus my understanding of what it should do. That only catches implementation errors. But often a much bigger problem is errors of understanding. Integration tests catch both.

In addition, there is a dramatic cost difference; if you're making intensive use of unit tests, it's not uncommon for them to outweigh all the rest of your code put together. And they need to be maintained, just like the rest of the code does. Integration tests are vastly cheaper -- and in most cases, you already need them anyway.

There are rare cases where it might be necessary to use unit tests, e.g. for internal error handling paths that can't be triggered if the rest of the system is working correctly, but most of the time, integration tests alone give better results for far lower cost.