views:

127

answers:

4

In .NET, TypeMock Isolator and Microsoft Moles allow one to isolate any class, property, or method - be it sealed, static, protected or non-virtual. So what was impossible to mock in Moq or Rhino Mocks, now no longer is the case.

I've always had some aversion with the idea of using an interface merely to be able to allow for mocking, when otherwise just the concrete class would exist. I'm not alone in this view (see here, here, and here). In the later it's implied that 'modern' mocking frameworks no longer need interfaces for testing or dependency injection.

However, while I can't speak for TypeMock Isolator, I can say that using Mocks in Microsoft Moles is extremely slow. Having code like the following in unit tests makes the speed of the test too slow to be used often:

MFile.ReadAllLinesString = (s) => csvDataCorrectlyFormatted;
MDirectoryInfo.AllInstances.GetFilesString = (di,fi) => fileInfoArray;
MFileInfo.AllInstances.NameGet = (fi) => "Doesn't Matter";

I'm sure that if the method being tested was programmed to an interface or abstract base class (so that the file system code could be abstracted away in a wrapper of sorts) that using frameworks like Moq for stubbing or mocking would end up being faster. But then we are back to the situation of having added production code complexity for basically the ability to unit test.

I'm leaning to the opinion that Isolator and Moles should only be used when one can't mock with traditional mocking frameworks. Yet, I still struggle with notion of having added production code complexity for the sake of testing.

I'm curious what the rest of the community thinks.

UPDATE 10/06/10: By saying I struggle with the notion of having added production code complexity for the the sake of testing, I'm referring to the adding an interface (or abstract class even) when otherwise one isn't needed, such as when making the concrete class non-sealed and methods virtual would do. The latter still allows seams for testing. Even if later one finds the need to use an interface for multiple implementations, couldn't they extract it from the class? But unless that need arises, why not follow YAGNI.

I'm all for the SOLID principles where they make the architecture of a program easier to maintain. I don't think these principles need to be followed religiously in every case. I think the mantra, "It always depends", comes into play many times. Otherwise, one is left with every concrete type having an interface or abstract base class, even when there will be only one implementation.

Lastly, I'm not saying that because Isolator and Moles allows one to get around isolation limitations in dynamic-proxy-based frameworks that one shouldn't still design architecture to be maintainable. In many cases, the SOLID principles are what is best, and thus Isolator or Moles wouldn't be needed. It's the cases where the interface is used solely for testing that I am questioning. I'm bringing up another side point about speed too. If one chooses to use Isolator and Moles it appears to bring a speed penalty. So I certainly don't think that they make dynamic-proxy-based frameworks obsolete.

+3  A: 

Irrespective of unit testing coding to interfaces helps provide a loosely coupled system allowing you to slide modules in and out relatively easily be it swapping one for the other or some sort of configuration.

That aside there are many, my self included, that would say that TypeMock and the like that allow you to mock statics allow for bad practices. Others would say we live in ivory towers and that TypeMock allows you to get on and get things done. I don't hesitate to think that perhaps both are right and wrong depending on your point of view and what you want to achieve. I myself will continue to use interfaces and RhinoMock, NSubstitue or Moq at a push.

As you can see from these posts Good Design is not Subjective and Design And Testability from 2008 to 2010 it is still a hughly debated subject as you can see from the comments.

Bronumski
I agree about the loose coupling, but what about when that isn't needed, when the interface has only one implementation, and thus the interface is there just for mocking?
Matt Spinelli
@Matt Open-Closed principle, enough said ... +1 to this answer
eglasius
@ Eglasius As has been mentioned before, the Open-Closed principle doesn't necessarily mean one has to use an interface. What about cases where public methods are virtual. This too would allow the class to be open for extending but closed for modification.
Matt Spinelli
+4  A: 

This is indeed an interesting question, which I sometimes heard from testing newbies. In my opinion, Moles/Typemock are not really comparable to mocking frameworks, and they are surely not a replacement. Here are my top arguments for it:

  • While it's worth every penny, Typemock comes with considerable initial license costs (~$800 per seat), why some managers hesitate to invest this amount of money. Moles on the other hand is free, but requires quite some work to set up a Moles assembly and comes with a quite strange 'API', which is very hard to learn - especially if you are totally new to testing.
  • Typemock and Moles are not 'Mocking frameworks' in the strict sense - they are some sort of profilers, which use runtime instrumentation under the hood. Thus, they come with a quite hefty performance impact. I have no concrete figures on that, but from my own experience I'd estimate that instrumented tests are around 100-400% slower (Typemock seeming to be faster than MS Moles)! With a large test battery containing thousands of tests, this may be an impediment in itself.
  • Last not least, the killer argument: A mocking framework forces you not only to use interfaces, but forces you to apply a highly decoupled and maintainable design on your software (along the lines of SOLID). - And remember: It's not the initial writing of the software, that primarily accounts for its costs, but it's the maintenance! - With Typemock or Moles, you basically can do whatever you want, and this really is a bad thing and the main argument against developing with Typemock or Moles.

Conclusion:

  • If you want to cover a brownfield project with tests, which was not written at all with testability in mind, then you have to use one of Typemock or Moles (in my opinion, Typemock is the better choice).
  • If you're starting a new project (i.e. you're doing a greenfield project), then use a mocking framework (I personally like Moq because of its simplicity) and use MS Moles as a complementary for things you can't have an interface for (e.g. .NET framework types).

Even Roy Osherove, the lead dev of Typemock, puts it that way (hopefully he's not getting into trouble with his boss because of this): A customer switches to MOQ – and I’m happy

There's one exception, though: If learning Test-driven development from scratch, it may be beneficial to use Typemock even for new code. This way, you can significantly flatten the TDD learning curve...

Thomas

Thomas Weller
I'm all for using Interfaces when the architecture of the project would benefit from loose coupling (i.e. layers, DI w/ IoC containers, AOP, etc). What I am referring more to with regards this question is when the architecture doesn't call for an interface and the only reason for adding one is for mocking. I know you could say that I still have more flexibility, but what about the YAGNI principle?
Matt Spinelli
Well, for me personally there's always to sides to source code: Production code and test code. Code without at least one covering test is just not complete, so YAGNI is just not an issue. The architecture may not force you to have an interface, but the ability to write test code does.
Thomas Weller
I want to vote up this answer, but the "If learning Test-driven development from scratch, it may be beneficial to use Typemock even for new code" part doesn't let me. I'd be afraid that doing that won't help the developer develop the design skills that you are encouraged to when doing TDD. I'd remove it altogether, since you already mentioned "If you want to cover a brownfield project with tests, which was not written at all with testability in mind, then you have to use one of Typemock or Moles"
eglasius
@eglasius - This is a pragmatic one, which I intimately know from my own experience. TDD with a pure mocking framework also requires considerable design skills. So if you want to introduce TDD to a team that has no pre-knowledge about SOLID software design, you force them to learn not only TDD, but also to aquire some design skills. In sum, this creates a very steep learning curve. In such cases, you desperately need a way to decouple TDD from software design skills. See also here: http://weblogs.asp.net/rosherove/archive/2008/09/26/unit-testing-decoupled-from-design-adoption.aspx
Thomas Weller
@Thomas I can see why.Personally, I continuously see a lot of third party code, and it scares the hell out of me the thought that nothing will change if they did unit tests ignoring the design aspect of it (which to me is precisely the point). I strongly agree with http://weblogs.asp.net/rosherove/archive/2008/09/26/unit-testing-decoupled-from-design-adoption.aspx#6643680, I've seen it plenty of times (also myself ...). I strongly disagree with the dismissal of valid concerns done here: http://weblogs.asp.net/rosherove/archive/2008/09/26/unit-testing-decoupled-from-design-adoption.aspx#6648296
eglasius
@Thomas A much clearer and thought out answer than mine. My only difference of opinion would be letting new TDD developers loose on a green field project with TypeMock as it will allow them to pick up bad habits quicker. I understand your argument above but with proper coaching, pairing and code reviews this shouldn't be a problem as I have seen it work. Anyway +1.
Bronumski
@Bronumski I agree with you that learning TDD with Typemock leaves aside the design aspect, but if you're teaching TDD to experienced programmers, they may not also learn proper design at the same time, but they will largely continue programming the way they did before - which usually is not that bad anyway ;-). But forcing them to learn TDD and software design at the same time may lead to a total disaster in some occasions - e.g. because you're putting a too high learing pressure on the devs which they might just not accept, or some limitations by the management (i.e. schedules) hinder it...
Thomas Weller
+1 for the link to Roy Osherove's blog post, and for his conclusion: TypeMock can help you learn to write tests, but it won't help you learn good design. If everyone on your team is a newbie at testing, TypeMock could be a valuable starter tool; if everyone is already a superstar at both testing *and* design, then TypeMock could be a great power tool. But MoQ forcibly pushes you toward loose coupling and good separation of responsibilities (even if sometimes it is extra work), and if your team aren't all design superstars, a tool that steers you toward the "pit of success" is no bad thing.
Joe White
+2  A: 

Not at all. From a purely technical point of view the interfaces are not strictly necessary. The only requirement for a mocking framework to construct a mock is that a method is virtual and the type has an accessible constructor. I utilize it quite a bit when I know that a particular class will not have multiple implementations.

I prefer doing TDD with a normal Castle Dynamic Proxy based mocking framework. It is all about creating a loosely coupled system with appropriately placed testing seams. Those testing seams could in future become extensibility points. The requirements placed by mocking considerations reinforce good design practices, causing more focused classes to be written.

Igor Zevaka
+1 "The requirements placed by mocking considerations reinforce good design practices, causing more focused classes to be written"
eglasius
A: 

But then we are back to the situation of having added production code complexity for basically the ability to unit test.

But that's not the reason you do it. You do it because otherwise the code has a hard dependency to an external element. Like a very simple specific implementation of a mail sender.

You isolate the external functionality behind simple contracts, that makes the rest of your code simpler / not more complex. You very clearly see the dependencies the code has, instead of those being hidden in the internals of the code.

You now have a simple mechanism to change those dependencies. There is less resistance when you end up needing to modify those contracts, that when the code is all mixed up in a chatty conversation in the internals of the class.

This is something that's hard to explain in the context of a simple stackoverflow question, specially as it usually takes a very long time for good design concepts to really sink in (if they do ...). I suggest reading about SOLID here.

Replacing like that with TypeMock has its place. To test legacy code that's not properly design, and maybe a handful other scenarios.

eglasius