Type of Projects where Unit Testing is useless

Question

Do you believe that Unit Testing (and test driven development) must be done under any set of circumstances or should there be some exceptions. I've been working on the type of projects lately where I can't see how Unit Testing would be useful or improve design, quality of code, etc. One type of project is PDF reports generator which takes aggregate data (values already calculated and QAed) and outputs it to a PDF report file. Another type is straight-forward CRUD applications using 3rd party ORM tool. I can see how someone could make an argument for using Unit Testing for CRUD application, but it's a lot of unnecessary and time consuming setup work, like stabbing out all the calls to a database and mocking business objects, etc. when at the end all you need to know if something happened to the DB. So when one should use or avoid Unit Testing?

Thanks

Answer 1

I've never really came across a strategy for unit-testing the view layer of the application that was worth the trouble.

I'd love for someone to prove me wrong about that, though.

Answer 2

I'd say never, simply because people may already have a tendency to perform less unit testing than necessary. If we say "unit testing may not be necessary" then we open the door for all kinds of excuses...

Answer 3

To paraphrase Jeff & Joel's recent comments on the SOB: you should do unit testing when it adds value to your product.

In the two cases you describe, you would be able to do a WHOLE LOT of ad-hoc fixes (and/or post-facto regression testing) for the same cost as writing unit tests. So don't write unit tests.

Answer 4

It often depends mostly on schedule constraints and what are you going to do with the code. If you're writting one-offs(which many would argue aren't existing anyway) I think unit tests are not needed.

However if you're writting a modular software that will be potentially re-used or re-factored in the future - unit tests are very helpful.

I often find the main purpose of them not to determine bugs in current implementation but to insure that no bugs introduced when changes arrive.

Answer 5

I agree with Ryan. The only usefull form of unit tests are the tests that are testing some algorithmic behaviour, which is usually a small part of the program.

If you find yourself writing usefull unit tests, then your job is probably not something usual :)

EDIT: On the contrary, integration tests are usually very usefull, but are much harder to automate then unit tests.

Answer 6

Avoid unit testing if the value returned by the testing is less than the cost of developing and executing the unit tests. This often boils down to risk analysis and expected life time of a piece of code.

Answer 7

"... it's a lot of unnecessary and time consuming setup work, like stabbing out all the calls to a database and mocking business objects, etc. when at the end all you need to know if something happened to the DB."

Sounds like you're testing the wrong thing.

Why stub out database calls? Why not create a test database and test the actual database calls against that test database? [That's the way the Django testserver works, and it is great.]

Why mock out the business objects? That's what you're testing, right? It's okay to say "we assume the ORM works" and declare your "unit" is the business object + ORM + database. Test that, since that's where the questions arise.

Unit test the things you wrote.

The things you downloaded should have their own unit tests.

Unit tests can come in layers -- you can write integration-like tests that assume that the ORM layer already passed all of its tests.

Answer 8

I am working on a software that interfaces with a production machine (using some industry standard as the communication protocol). As a result, most methods of the communicator class are triggered by messages from the machine, they are never called from the outside. So I made them private.

If I want to test this functionality, I need to create a simulator that simulates the machine behaviour, in order to run some black box tests on this class. Building the simulator would probably take at least 50% of the time I needed for the communicator.

I would have made the functions that are triggered by the machine public, but that started some very heated arguments in our team...

Answer 9

Please don't conflate unit testing (UT), which is a tool, with test-driven design (TDD), which is a methodology. see this for more

Also, remember that one of the benefits of unit tests is for regression testing, i.e. as insurance against future breaking changes (and to help verify that fixed bugs do not become un-fixed).

In your first example, I would unit-test the PDF conversion as follows:

• start with a known set of sample data, and generate the PDF document
• manually verify that the generated PDF is correct
• retain the sample data and correct PDF document as part of the test suite
• write a function to compare two PDF documents for differences (at the binary or content level, as appropriate)
• automate this unit test to compare the PDF document generated from the sample data with the verified-correct document from the first step

Now, whenever you change the PDF generation code, you have an automated unit test to verify that you did not break document generation at least for the sample data and output format.

I have used the same technique for verifying HTML-rendered pages, GUI screens, etc. Manually verify the baseline, then automate the comparison after that. This provides insurance against future breaking changes.

Of course if your output formats change frequently this may not be as useful, but a baseline test of the basic output formatting is likely to have a long lifespan.

In your second example it sounds like you would be testing the 3rd-party ORM tool, which is probably pointless. However, again some baseline tests may be useful as insurance against future breaking changes, depending on how you are using the tool. For example, instead of mocking up the entire n-tier chain (I'm not a fan of mocking when it is not necessary), just use the ORM tool to create, read, update, and delete a typical object/record, and verify the operation using direct SQL statements on the (test) database. That way if the 3rd-party vendor later updates something that breaks the basic functionality you'll know about it, and new developers to your project can easily see how to use the ORM tool from the unit test examples.

Answer 10

• For example writing automated tests for the user interface/presentation layer is often not worth the trouble. In that case it's often best to keep the presentation layer very thin, and have a testable layer which contains all the functionality of the presentation layer. Then the presentation layer will be mainly declarative and contain only simple glue code. When everything is declarative, there is little that can break, so testing it is not necessary. Things such as are the UI elements aligned properly, is the layout ok, are the labels right etc. are easier to test manually.

• Throwaway code. If the code will be used only once, and it is not important that it works right, and if ever the need to modify the code arises you can afford to rewrite it, there might not be benefit from writing high quality code and tests. But when the code gets bigger, or you need to maintain it, or it is important for the code to work right, writing tests is worth the effort. I once heard somebody say that if he writes more than 10 lines of code without tests, then he begins to be unsure whether the code he just wrote will work right.

Answer 11

I think you should avoid unit testing on projects where the code doesn't need to work.

Unit testing is extremely useful if you want the code to work properly now and when you'd like it to keep working in the future when there are modifications. On projects like that it more than pays for itself it time savings, but on all others it is needless overhead.

Answer 12

If your code is going to be reviewed or used by someone else then it should be tested. If not at a class level, then at some higher level. I hate when people send their code out for review when it is obvious that they didn't take the time to bother that it works (or worse yet doesn't even compile).

I seldom find unit testing (at class level) of any value. I can easily do that kind of testing by walking through the debugger when I'm developing the code. I prefer to test classes that are supposed to work together as a unit. I get way more bang for the buck taking this approach. After all, I don't really care if functionX does exactly what I think it is supposed to do when it is by itself, but doesn't do what it NEEDS to do when working with the other classes in the system.

Answer 13

To answer your question, there are definitely some exceptions.

In fact, I'd say the usefullness of automated white-box testing is more likely the exception rather than the rule.

It takes extra time, the tests have to be rewritten anytime there is significant change, slowing down and hindering development. Also, just becuase your tests pass is no guarentee that your app is working correctly or solving the problem it is intended to for users.

White-box testing with automated tests has been a tool in the toolbox for awhile--lately for some reason it is all the rage, and if you speak out against it you are ostracized to a degree by TDD zealots.

Better to understand it, and use it where it could be useful. For example, a friend of mine was an SDET on the Microsoft Frarmework team -- this was a gerat example of where TDD was a great asset. That is, they were developing the engine for a development environment to be used by millions of people.

For your run of the mill web app, even enterprise level, I would be skeptical that such rigorousness is worth the added time and complexity. However, if you have a key component that will be used/re-sued extensively, it could be worth it for those pieces, etc.

Answer 14

"I once heard somebody say that if he writes more than 10 lines of code without tests, then he begins to be unsure whether the code he just wrote will work right."

This person is a danger to his or her self. Remove them from the code immediately.