Should I write unit test for everything?

Question

Hi,

I am wondering should I write unit test for everything. There are some classes is very difficult to write unit test. For example, I am writing some program for handling audio. The class for capturing audio from microphone, and class for play audio to speaker, how can I write unit test for those classes? I can't get output and input of those classes, so it is almost impossible to test them? The only test I can do is for getter and setter, those boring test. So the question is, what is the guide line for writing unit test? And how should I deal with these classes are difficult to test?

Thanks. Victor Lin.

Answer 1

If you're facing difficulties setting up a particular area of code for testing, it might be worth investigating a mocking framework like jMock, or EasyMock.

Answer 2

Use unit testing where it makes sense - don't aim for 100% coverage. The main guideline is to think rather than applying either dogma or laziness.

Having said that: If you have classes which are naturally hard to test, try to reduce how much they have to do. Isolate the untestable code and separate its API into an interface. Then test the logic which uses that API against a mock or stub.

Answer 3

The short answer is, No, but then that goes for everything in programming when you ask, "Should I X for everything?"

The longer answer is you should at least consider it, which you are doing. Why can't you mock input into a sound recording class? Why not have the class bypass recording from a microphone to load audio from a file and have the output go to a file instead of a speaker. The test could compare the output result to the known correct result.

For more on the philosophical side see this.

Answer 4

To answer your specific question, use a loopback cable. Connect the speaker out to the mic in. Write a test that renders to the speaker and captures from the mic and verifies that the same thing you played was captured. My suggestion is to use a simple sine tone so that an FFT can tell you if you captured back the same thing.

The answer to the more general question is yes, you should unit test everything you can. Doing so creates a legacy for later so changes down the road can be done with peace of mind. It ensures that your code works as expected. It also documents the intended usage of the interfaces. Finally, it forces better coding style. Usually something which is hard to unit test is also poorly designed. Writing for testability means writing for better design.

Answer 5

I only write unit tests where I know it saves me time. When I started out unit-testing this was only a small percentage of the classes (those awful ejb's!!). Today I test almost everything and I save total development time on every single thing I do. If there was an efficient way of testing user input through a microphone I'd do it too. But as far as I know, it's not possible in a manner that saves me time.

So i think you should unit test everything that is in your current "test capability". You should try to stretch this capability, but overreaching really sends signals of wrong priorities; In all likelihood there's some other test that deserves your attention more. (Technically I'm test infected but not TDD infected)

The law of diminishing returns applies to unit testing as much as any other testing; your payback on that last 5% is very low and the cost is high.

Answer 6

One rule I use for deciding whether to develop unit tests: if your class (or whatever unit) is going to be released "into the wild", then you should definitely consider writing unit tests.

By "in the wild", I mean: once your code is going to be in a situation where you cannot predict or control how things will interact with it. So classes exposed through an API, or classes exposed to user input should probably be unit tested.

Personally, I think that writing unit tests for everything is likely to be a waste of your time. It really is a complex issue where you need to consider:

• How complex is the class? Dead-simple classes might not be worth testing.
• How critical is the class? Code running on an ATM would hopefully be unit tested.
• How much control do you have over how the class is used?

Then there's always:

• When is the project deadline?
• How much man-power do you have devoted to creating tests?
• Are your requirements concrete and detailed, or is the class still changing fairly often?

As for difficult classes, maybe do some reading about fuzz testing.

Answer 7

Other example: If you develop a game engine, you want to test your shadowing and other functions, but you have to confirm it visually - that's nothing a classic UnitTest can decide.

Your Case: As your application becomes more complex it gets painful always to click through your UI to test all your functions.

I would write an "interactive TestSuite", where various dummy dialogs (customized for each test-case) are shown with only the functions you need to test. Once you close the Dialog, you will be prompted if the behaviour was expected. But I am not sure if there are solutions available which could help here.

Answer 8

You might want to check my series "Testing the Impossible" in my blog for some ideas how to test.

In your case, I suggest to follow the idea of Steve Rowe to write a test which sets up the speaker and microphone and use a loopback cable to test the hardware setup code plus the API which allows to emit data via the speaker and read data from the mic.

This is a unit test but not an automated one. Move it into an independent test suite which doesn't run with the other automatic tests. If you want to automate it, set up a second PC with the correct configuration (plus the loopback cable) and run the test remotely.

After that, you are sure that the hardware setup works, you can send and you can receive audio. This allows you to test the data processing classes independent of the hardware. Use mockups to simulate the speaker and mic.

Answer 9

Code that capture and replay audio cannot be unit-tested (albeit you could test that the capture method returns an error when called when the class isn't successfully bound to a resource).

However, unless you simply write the captured sound to disk the code that invokes your capture and play classes certainly can.

Two pieces of advice:

• Don't test the compiler (e.g. getter and setter)
• Test everything that could possibly break

Answer 10

I tend to write tests as much as I can. Not only to prove that something works, but to make it obvious to someone else when they inevitably break it later.

I've had a method that was 1 line long. In similar places in my app I had written unit tests, but being in a rush I thought that it could not possibly fail. I was wrong, it didn't work :-)

This site explains when you should unit test: http://whendoitest.com/

Answer 11

Cheap answer: Test everything that could possibly break

Ultimately though you need to understand the business value of the tests you write - no different to any other effort you expend, any other codebase you commit yourself to maintaining.

Answer 12

Designing tests is an acquired skill - the more you test the better at it you get. Some things appear hard to test but if you think about it for a few minutes you can often find a way.

Tests can be messy - e.g. a java program can launch an interpreter or even a shell such as bash which in turn launches a series of unix filters which you hope will output identical binary files. Don't worry though - the quality of test code does not have to be as high as the code in the finished product.