A few questions about unit tests

Question

Two questions about unit tests.

1. I've been writing unit tests for a while, however they're usually to test classes I already had written. Recently I read an article (mind you an old article) that says you should write unit tests before you begin writing your code.

   Does anyone actually follow this methodology? It seems like a good idea on paper, but in practice is it?

2. Should you write unit tests to see how your method handles bad/malicious input? Obviously you would want to write tests against functions which are specifically meant to handle "user" input to see how it handles bad/malicious input, but what about functions which should never have this type of input passed to them? At what point do you draw the line?

Answer 1

The methodology of writing unit tests before the classes is called Test-Driven Development (TDD) and was popularized by Kent Beck in the early 2000s. The idea is that you write a test that describes the functionality that you need. Initially, this test will fail. As you write your class, the test passes. You refactor your test to add more desired functionality, then refactor the class to make this new test pass. Your class has met its goals as soon as the tests pass. Of course, this scales up beyond classes as well.

As to what types of tests to write, it depends on if you are testing a public API or a private API. Public APIs should have more extensive tests written to ensure that input is well formed, especially if you don't fully trust the users of your API. Private APIs (methods that are only called by your code) can probably get away without these tests - I would suspect that you can trust your own development team to not pass in bad data to them.

Answer 2

Writing the unit tests first is quite a common practice. A big benefit is that you do not just write tests that your code will pass but rather tests that define what is important, what you are trying to achieve, and what you want to make sure will not happen. It can help you flesh out your design. Also, you can vet the speck with external stakeholders before you code.

As for what tests to write, that is a bit subjective based on the time you have. I would not go crazy vetting code for scenarios it will never face. That said, it is amazing what input makes it to code that "will never see it". So, more tests are better but there are definitely diminishing returns at some point.

The language you are coding in matters. Dynamic languages require more tests because the compiler will catch fewer issues and bugs may be harder to track (since they can propagate further from the initial input issue). At least, this is my opinion.

It also makes a difference where the input is coming from. The general public should be considered positively malicious (ie. the web), employees should be assumed incompetent, and even fellow coders (and yourself!) should be assumed to be at least careless. But the danger falls as you get closer to your inner circle.

Answer 3

Test Driven Development is a pretty widespread concept. The basic idea is that you are trying to only write code that is necessary to satisfy some requirement for the software. So, you write a test for the requirement, and then the code to make the test pass.

I personally don't use TDD, but I know people who do. My personal thoughts are that it is very useful if you're working on something that is more application-driven, like a database or user interface. But, for something that is more algorithm-heavy (like a physics model), I find that it breaks my train of thought and gets in the way.

Answer 4

Does anyone actually follow this methodology?

Yes.

It seems like a good idea on paper, but in practice is it?

Yes.

Should you write unit tests to see how your method handles bad/malicious input?

Yes.

What about functions which should never have this type of input passed to them? At what point do you draw the line?

When it moves from software to psychosis.

You can -- if you want -- write tests for impossible situations. However, you're wasting your time and your employer's in an obvious way.

You write tests for the defined use cases. And that's it.

You do not make up random test cases based on your imagination.

What If? What if the defined use cases are incomplete? Bummer. You write tests for the official, contractual, public interface -- and nothing more.

What if the design is inadequate and you realize that the given interface is riddled with incomplete specifications, contradictions and security holes? This has nothing to do with testing. This is just programming. Bad design is bad design.

What if some malicious sociopath takes your code and uses it in a way that exceeds (or otherwise fails to meet) the defined specifications? Bummer. The sociopath wins. They were able to put your code in the impossible situation you didn't test for. Buy them a beer for being so clever.

Answer 5

There's already a quite a good bunch of answers but i'd like to say one more think to the question number 2.

If you make your unittest code "data driven", it shouldnt matter if the code is testing "bad" or "good" input. What matters is that you have big enough data set that covers both.

Answer 6

There is a difference in mindset between test-before and test-after. Writing tests before is a form of design, in that you are designing your code's interface and defining expected behaviour. When you then write code that passes the tests, that validates your design. And at the end of development, you happen to have a suite of tests already in place!

With test-after, you need to be careful to avoid the trap of writing tests that your existing code will pass. It's a different focus, and you don't get as much out of it as the test-before version.