TDD: Does it get in the way of good API design?

Question

I've never written TDD code, but I've seen a decent amount of discussion about it here on SO. My biggest concern with it is that it seems like general good API design (for flexibility, ease of use, simplicity of interface, and performance) takes a back seat sometimes to making code mockable, ultra-modular beyond what is necessary for any API use case, etc. For example, TDD proponents often suggest that things be passed in as parameters that, from an API abstraction perspective, the method being called should "just know", or that classes and methods be factored in a way that makes testing easy, which is not necessarily the way that relates best to the problem domain.

To people more experienced with both TDD and API design: Do you find that TDD often gets in the way of good API design? If so, how do you counter this?

Answer 1

No, I find that TDD generally encourages good design. Things which are easily testable are often easy to use in production too... because when coming up with a test, you think, "What would I like to do next?" and then make it work.

When practising TDD you are forced to think through the 'use cases' for an API, how the programmer is going to use the classes/methods and you will end up with a very usable framework. If you have to create a hundred FooFactory and BarAuthenticators, or the API is becoming 'ultra modular' like you say, you will likely pick up on that while writing your test code and think about how to simplify it.

As for parameters and dependency injection - I usually find that dependencies become a lot clearer using TDD. They're usually constructor arguments rather than method arguments, but making it clear that an API implementation needs an authenticator, a source of randomness etc. is useful in understanding what it's doing. You can be sure that the code isn't going to hit the network or a remote database somewhere because it's hiding these things in its implementation. These kind of hidden details are what makes an API hard to use in a test environment.

Note that when dependencies are part of the constructor call, it's not part of the interface that class may implement - the dependencies within an interface are usually hidden, but TDD means that the implementation exposes them in an obvious way.

A good source of information about dependency injection and TDD can be found on the Google Testing Blog. Pay particular attention to the posts by Miško Hevery or watch some of his videos on Youtube: Don't Look For Things and more.

Answer 2

Most of the things you have listed as disadvantages of TDD are considered by many to be elements of good design.

If particular note the idea of something being passed in instead of something the method should "just know" - the latter often leading to singletons or other anti-cohesive designs.

While it may be true that you may end up with a design where some aspects are only there to support testability this is not necessarily a bad thing.

As a rule, though, if you have to rethink your design to make it testable you will general arrive at a design that is better in terms of cohesiveness and fitness for purpose - while still being flexible in the sense that it can easily be changed to whatever your future needs are through refactoring because you have the tests there to give you the confidence to make changes quickly.

Answer 3

TDD leads to emergent design which ultimately produces a very usable and extensible API. TDD isn't about testing. It's about creating seams in your code that you can use to add behavior as you discover more about your project.

Answer 4

The point about TDD i sthat we write tests which call the code as we design it. Consequently we should end up with an extremely usable interface. Of course it doesn't entirely happen by chance. We still have to make the right choices.

Answer 5

You might want to take a look at the newspeak language work by a.o. Gilad Bracha to see some relevant API and language design.

Answer 6

With traditional API design, it's easy to build yourself into a corner: You can end up with an API that has lots of hidden dependencies (like class A needs B needs C needs D and if you change the order in which the classes are initialized, things start to break).

TDD makes sure you that separate pieces stay separate. It also allows you to see your API from a very unusual perspective: As a user/consumer. Your first question is "How do I want to use this?" not "How do I want the API to look like?" The latter can lure you into a hidden trap while the first leads to something which I call "intuitive API": It behaves as expected.

A word of advice: Don't make TDD your religion. It's a tool and some problems can't be solved with some tools. So if TDD doesn't work for you for any reason, then that's OK. Use as much TDD as you like. Don't be fanatic about it. Over the years, you'll find your comfort zone.

Answer 7

I haven't noticed that TDD makes for bad API choices at all.

On the contrary, by creating a clear framework in the first place the API is also more clear, and more obvious to its users.

Answer 8

I use TDD and think it works well with respect to API construction, but I think that APIs, especially ones that you expose to external customers, are an area in which you need a little bit more upfront design than typical when using TDD. TDD relies as heavily on refactoring as it does on test-first. With an API, you don't always have the luxury of both refactoring your method signatures AND keeping a clean design. If you aren't careful about designing your interface upfront you can find yourself with an ugly interface that supports multiple methods doing very similar things just to keep backwards compatibility with your existing API while at the same time moving your design forward.

If I have an API that I know I will be exposing to external users, I generally take more time to think about the signature and get it as close to "right" as I can before starting the development process. I also expose it early and often to my customers to get feedback on it so that we converge as quickly as possible to a stable interface. Refactoring behind the scenes to improve the implementation without changes to the interface is not so much a problem, but I want to get a stable interface quickly and am willing to invest more upfront to get it.

Answer 9

I agree 100% with the top answers. But really this depends on what you mean by "good API design". TDD leads to testable, modular, working code-- but in the end, the most important aspect of TDD code is the testability.

You will find that it leads to different design than a other processes yield. You will find that testable code may expose a few more bits of its internals than a pure API might expose.

In many cases it may make sense to use TDD to build working code, and then-- as a separate step-- pull out the API. Otherwise you have two forces working somewhat in conflict: a simple API and testable code.

This is a subtle point, though, and overall TDD will provide much better APIs than designing from an ivory tower.

Answer 10

I've been using TDD for several years now and I find that it drives an API design towards a more usable design by providing you with two different clients for the API from the start; you have production code and test code both of which want to drive the API in different ways.

It's true that sometimes I add things for the sake of making it easier to test the API but I almost always find that things that I think I'm putting in just for testability's sake are actually very useful for monitoring purposes. So, for example, a FooAllocator might end up with an optional constructor argument which is a monitoring interface (IMonitorFooAllocations) which is very useful for mocking out during testing to enable me to take a peek inside but which also tends to be very useful when you suddenly find that you have to expose some allocation metrics to the rest of the world whilst in production. I now tend to think of the extra bits that I might want to add to enable easy testing in terms of their dual use for optional production monitoring. I generally write server code and being able to expose the internals of things as perfmon counters is VERY useful...

Likewise you're correct to say that often the objects that makes up an API might take some other objects explicitly rather than reaching out and getting them from a known place but this is a good thing. Trust me, once you get used to dealing with explicit dependencies you wont want to go back to having to dig through class after class to work out how and why your Widgets are accessing active directory when there's no hint in the API that they would want to do such a thing. Also it's often the case that you break open these dependency relationships during design and testing and then hide them again as you put all of the pieces together. You still 'parameterize from above' but more often than not an API's object model might mean that you never really see the 'above' as a user of the API. You end up with one place to configure the API with the things that it needs and often this looks no different to how it would have looked if you had a mass of singletons and globals and hidden dependencies.

But remember, TDD is a tool, when it doesn't fit don't use it.

Answer 11

On the contrary, since you are using your methods thinking as a user by writing your tests, you will spot the issues that your user will come up with as you write it. In fact, you will end up with a much cleaner and user-friendly interface that what you would have written otherwise.

Answer 12

If your API is suffering due to internal requirements of your objects, that's what the facade pattern is for. Not all objects need to be public.

A lot of the pain points of TDD are, in fact, signs of pain points in the design. If it's hard to create a class because it requires 18 dependencies to be passed in, the fundamental problem is that the class has too many dependencies and is going to be quite brittle. It also probably does waaaaaay too much. The "TDD pain" in this case is a good thing, as it makes the other issues more obvious.

Answer 13

TDD is an API design technique. Every time you write a unit test, you are either creating an API for the first time, or using a previously created API. Each one of those tests lets you "feel" how easy or difficult the API is to use. Each new test forces you to consider the API from a new point of view. There can hardly be a better way to design APIs than to exhaustively use them the way TDD forces you to.

Indeed, this is the reason that TDD is considered a design technique rather than a testing technique. When you practice TDD, you do not design your APIs in a vacuum. You design them by using them!

Answer 14

TDD doesn't get in the way of good API design.

Mocking gets in the way of good API design.

The two are not synonymous - I independently started using TDD before others wrote the books on it, simply because it makes clear whether requirements are testable and designs meet a requirement.

However, I think Mocking is bad because it affects the design and implementation and introduces other, artificial requirements. It also exposes internals of an implementation that shouldn't be, and makes the testing brittle. It tests how something is done, rather than what is done.

How to counter it: Use TDD, but don't use mocking.