views:

317

answers:

4

The test class below verifies that a simple HttpService gets content from a given URL. Both the implementations shown make the test pass, though one is clearly wrong because it constructs the URL with an incorrect argument.

To avoid this and correctly specify the behaviour I want, I'd like to verify that in the use block of the test case, I construct one (and only one) instance of the URL class, and that the url argument to the constructor is correct. A Groovy enhancement seems like it would let me add the statement

mockURLContext.demand.URL { assertEquals "http://www.foo.com", url }

but what can I do without that Groovy enhancement?

Update: Replaced "mock" with "stub" in the title, as I'm only interested in checking the state not necessarily the detail of the interactions. Groovy has a StubFor mechanism that I haven't used, so I'll leave my code as is, but I think you could just replace MockFor with StubFor throughout.

import grails.test.*
import groovy.mock.interceptor.MockFor

class HttpServiceTests extends GrailsUnitTestCase {
    void testGetsContentForURL() {
        def content = [text : "<html><body>Hello, world</body></html>"]

        def mockURLContext = new MockFor(URL.class)
        mockURLContext.demand.getContent { content }
        mockURLContext.use {
            def httpService = new HttpService()
            assertEquals content.text, httpService.getContentFor("http://www.foo.com")
        }
    }
}


// This is the intended implementation.
class HttpService {
    def getContentFor(url) {
        new URL(url).content.text
    }
}


// This intentionally wrong implementation also passes the test!
class HttpService {
    def getContentFor(url) {
        new URL("http://www.wrongurl.com").content.text
    }
}
A: 

What exactly are you expecting to have fail? It is not readily apparent what you are trying to test with that code. By Mocking URL.getContent you are telling Groovy to always return the variable content when URL.getContent() is invoked. Are you wishing to make the return value of URL.getContent() conditional based upon the URL string? If that is the case, the following accomplishes that:

import grails.test.*
import groovy.mock.interceptor.MockFor

class HttpServiceTests extends GrailsUnitTestCase {
    def connectionUrl

    void testGetsContentForURL() {
        // put the desired "correct" URL as the key in the next line
        def content = ["http://www.foo.com" : "<html><body>Hello, world</body></html>"]

        def mockURLContext = new MockFor(URL.class)
        mockURLContext.demand.getContent { [text : content[this.connectionUrl]] }
        mockURLContext.use {
            def httpService = new HttpService()
            this.connectionUrl = "http://www.wrongurl.com"
            assertEquals content.text, httpService.getContentFor(this.connectionUrl)
        }
    }
}
laz
Hmmm. I follow his question but not your response. He is trying to mock the URL class, not his HTTPService. The HTTPService is the code under test.
cwash
I removed that part of the comment. The reason I asked about it is because I am unable to tell what the original poster is trying to accomplish with the code and description provided here. The code I provided will pass the case that is expected to pass and fail the case that is expected to fail. What does it really prove?
laz
It's a simplistic example, but demonstrates some subtleties involved in state-based testing. What he is trying to verify is that his mock is set up correctly. In most mocking frameworks this is done by setting expectations on what the expected set of parameters are. Here he is trying to set expectations on the parameter to a constructor but can't do it because of the enhancement he mentioned.
cwash
To clarify, I meant interaction-based testing in my last comment... (not state-based) Sorry about that. Interaction based testing is about verifying collaborators talk correctly to each other (not necessarily that the end result of a method call is what you expect). This is why you want to set expectations on a mock. It ensures that the code being tested interacts with that object properly.
cwash
cwash is right. Another way to say the same thing is that I want to document the correct behaviour in the test, and that correct behaviour is "make a URL object using the string I pass you, and give me back whatever it says the content is". The test laz proposed doesn't do this (in fact, I've read it a few times and actually can't tell what behaviour it is trying to describe).
Douglas Squirrel
What I am hearing is that you want to test the validity of the mock URL object. What are the criteria for the considering the mock URL object to be valid? How do you define "whatever it says the content is"? Is the idea that you want to map a test URL string to a predetermined content string? The code I proposed allows you to map a test URL string to some predetermined content. Perhaps I might understand better if you indicate where would you use that line of enhanced constructor-mocked Groovy code within the rest of the code.
laz
My $.02 - it's less about having the URL map to predefined content. That data is there just to ensure that the interactions are correct. If you can show the interaction between the objects happens properly there's not much need to perform the same interactions with multiple data sets/scenarios.
cwash
If I don't hear anything else in the next 24 hours I'm going to delete my answer since I seem to have missed the boat on this one.
laz
mockURLContext.demand.URL { assertEquals "http://www.foo.com", url } would go right before the other mockURLContext.demand... line - by the way, the latter is exactly how you define "whatever it says the content is" and how you "map a URL string to predetermined content". Together, the two demands say what the class-under-test must do with the URL object it constructs (this is different in important ways from "validating the mock URL object"). Steve Freeman's book http://www.mockobjects.com/book/ is a good introduction to the type of behaviour-driven testing I'm doing here.
Douglas Squirrel
Don't delete your answer - the comments under it may be useful to others, and I'm glad to have heard from you.
Douglas Squirrel
Will do. Thanks for the link to the book, I'll have to check it out.
laz
A: 

It's tough to mock out JDK classes that are declared final... Your problem, as you reference through the enhancement, is that there is no way to create a URL other than calling the constructor.

I try to separate the creation of these kinds of objects from the rest of my code; I'd create a factory to separate the creation the URLs. This should be simple enough to not warrant a test. Others take a typical wrapper/decorator approach. Or you may be able to apply the adapter pattern to translate to domain objects that you write.

Here is a similar answer to a surprisingly similar problem: http://stackoverflow.com/questions/565535/mocking-a-url-in-java

I think this demonstrates something that a lot of people learn after doing more testing: the code we write to make things more testable is meant to isolate what we desire to test from what we can safely say is already tested somewhere else. It's a fundamental assumption we have to make in order to do unit testing. It can also provide a decent example of why good unit tests aren't necessarily about 100% code coverage. They have to be economical, too.

Hope this helps.

cwash
You correctly describe standard workarounds for Java's inability to mock new() calls, but these tend to obfuscate relatively simple cases like this one. I don't ever intend to use an alternative way to get URL content, so a URLFactory doesn't add anything to my design and in fact makes the code harder to follow (you have to look up and read URLFactory.groovy to be sure it doesn't do anything weird).MockFor avoids this in many cases, such as in the example test where I'm verifying the call to content() directly without injecting any factories. I just want the same benefit for constructors.
Douglas Squirrel
I disagree with cwash's last paragraph: here I know exactly what I want to test, namely HttpService (and _not_ the URL class). My implementations of HttpService are intentionally simplistic for readability, but you don't have to add much complexity to run the risk of a hard-to-find error arising from a wrong constructor call.
Douglas Squirrel
Agreed. I was just trying to answer your question with how you would go about doing this *without* the enhancement to Groovy. I didn't say it would be pretty. :)In pure Java code, you can use JMockit to accomplish the same thing. It can be a little tough to properly mock JDK classes with that as well, but is doable in much the same fashion.
cwash
Regarding your second comment - What is it you disagree with? That you need to sacrifice simplicity for testability? Whenever you mix the new operator in your business logic, you miss the opportunity to create a testing seam in that code. You tightly couple your dependencies. Of course using the new operator is more simplistic than injecting dependencies. But is the concept of inversion of control really that much more complex? The factory code then goes away and you allow a framework (that is already tested in its own right) to do this wiring for you.
cwash
Loose coupling and inversion of control are great for business objects - and Grails services are very nice mechanisms for injecting without factories - but I'm _never_ going to need to plug in a different URL implementation. So why should I confuse readers by implying (with a factory or service or whatever) that there might be some other reasonable implementation of the URL interface? To ask it another way, why don't we use an IntegerFactory or a StringService every time we want to write x=5 or y="foo"?
Douglas Squirrel
The reason is you want to test the code. So what is more important to you, the ability to test this construct, "to check that I haven't done anything really dumb and messed up the _construction_ of the URL object" - as you said in the comment above. My contention is that this isn't that easy for objects you don't have control over. You need to introduce the complexity involved in loose coupling and inversion of control if you want to test this on an isolated, unit-based level (without the construct your original question alluded to).
cwash
+3  A: 

What does mocking the URL get you? It makes the test difficult to write. You won't be able to react to feedback the mock objects give you about the design of the API of the URL class, because it's not under your control. And if you don't precisely fake the behaviour of the URL and what it exposes about the HTTP protocol, the test will not be reliable.

You want to test that your "HttpService" object actually loads the data correctly from a given URL, copes with different content type encodings correctly, handles different classes of HTTP status code appropriately, and so forth. When I need to test this kind object -- one that merely wraps some underlying technical infrastructure -- I write a real integration test that verifies that the object really does use the underlying technology correctly.

For HTTP I write a test that creates an HTTP server, plugs a servlet into the server that will return some canned data, passed the URL of the servlet to the object to make it load the data, check that the loaded result is the same as the canned data used to initialise the servlet, and stop the server in the fixture tear-down. I use Jetty or the simple HTTP server that is bundled with JDK 6.

I'd only use mock objects to test the behaviour of objects that talk to the interface(s) of that object I've integration tested.

Nat
You describe integration tests that verify correct usage of HTTP, which of course are very useful. But they will be slow to run, thanks to all that setup and teardown. I want to run thousands of unit tests in seconds, and only then run slower integration tests to get additional feedback. So how can I verify that I wrote the HttpService in a basically correct way, before going on to the more detailed (and important) integration tests you describe?
Douglas Squirrel
I realised thanks to this answer that I should have been referring to "stubs" in the question, not "mocks", as a simple recording of the calls and a state-based check would be perfectly fine (see http://martinfowler.com/articles/mocksArentStubs.html). I'll update the question.
Douglas Squirrel
The only way to tell if your HttpService runs correctly is if you duplicate the behaviour of the URL class and all the hidden infrastructure that runs behind it _absolutely exactly_. Do you really think you can do that? Do you know exactly how it is implemented? More to the point, do you really think it's worth the cost? And what if it changes in a future release?
Nat
As for your worries about performance: I have not found it to be a problem. I usually have far more unit tests of objects that are independent of technical infrastructure than integration tests of the adapter objects (like your HttpService) that sit between them and and the technical domains. The integration tests are narrowly focused. That means that the integration tests do not have a big effect on the overall time of the entire test run, especially compared to full end-to-end tests. If they do, modern CI servers make it easy to run test suites in parallel.
Nat
You are right that a full check of the HttpService requires proper integration tests (or duplicating the implementation, which is unrealistic). But I just want to check that I haven't done anything really dumb and messed up the _construction_ of the URL object, as in the wrong implementation above. Surely I should be able to do that? I am writing this service on my own machine prior to checkin, hence running the tests about once a minute (or in the background continuously. a la JUnitMax) - so the difference between an 0.1 second unit test and a 5-second integration-test is significant.
Douglas Squirrel
I don't know where you get the "5-second" measurement from. My tests that start an embedded HTTP server and do a request that makes the servlet query the database run in tenths of a second.If you want to test that your code can correctly instantiate a URL object, make it instantiate a URL object. It's the only way to know for sure.
Nat
To check, I just wrote integration and unit tests for a slightly more complicated class, a Grails controller that parses some XML downloaded from a URL. The two tests verify the same behaviour, but one stubs the URL request and the other runs Jetty with a stub servlet as you describe in your answer. The unit test takes 0.12 seconds, the integration test 0.72 seconds. 5 seconds was obviously wrong (based on my experience with more complex integration tests that use a database) but it's still a 6x difference in execution speed, so wouldn't be acceptable if you had a lot of these tests to run.
Douglas Squirrel
This was the most interesting and informative answer, though it didn't actually answer my technical query about mocking Groovy constructors. I'd still like an answer to that question!
Douglas Squirrel
+2  A: 

Putting on my "Programming in the Small" and "Unit test 100%" hat, you could consider this as a single method that does too many things. You could refactor the HttpService to:

class HttpService {
  def newURLFrom(urlString) {
    new URL(urlString)
  }
  def getContentText(url) {
    url.content.text
  }
  def getContentFor(urlString) {
    getContentText(newURLFrom(urlString))
  }
}

This would give you a few more options for testing, as well as split out the factory aspect from the property manipulation. The testing options are bit more mundane then:

class HttpServiceTests extends GroovyTestCase {

  def urlString = "http://stackoverflow.com"
  def fauxHtml = "<html><body>Hello, world</body></html>";
  def fauxURL = [content : [text : fauxHtml]]

  void testMakesURLs() {
    assertEquals(urlString, 
                 new HTTPService().newURLFrom(urlString).toExternalForm()) 
  }
  void testCanDeriveContentText() {
    assertEquals(fauxHtml, new HTTPService().getContentText(fauxURL));
  }
  // Going a bit overboard to test the line combining the two methods
  void testGetsContentForURL() {
    def service = new HTTPService()
    def emc = new ExpandoMetaClass( service.class, false )
    emc.newURLFrom = { input -> assertEquals(urlString, input); return fauxURL }
    emc.initialize()
    service.metaClass = emc
    assertEquals(fauxHtml, service.getContentFor(urlString))
  }
}

I think that this makes all the assertions that you want, but at the cost of sacrificing test readability in the last case.

I would agree with Nat about this making more sense as an integration test. (You are integrating with Java's URL library on some level.) But assuming that this example simplifies some complex logic, you could use the metaclass to override the instances class effictvely partially mocking the instance.

jkl
+1 - This seems to be an alternative to the original proposed method using the groovy enhancement. I think it makes sense to say that often your approach to make things more testable will come with a set of tradeoffs that need to be considered. Tests need to be thorough, but economical.
cwash