Testing Applications built on top of RESTful web services

Question

Lets say; I am developing a Web Application which talks to a RESTful web service for certain things.

The RESTful web service isn't third party, but is being developed parallely with main application (A good example would be, E commerce application and payment processor; or Social network and SSO system).

In such a system, acceptance(cucumber) or functional tests can be done in two ways:

1. By mocking out all external calls using object level mocking library; such as Mocha or JMock.
2. By doing mocking at http level, using libraries such as webmock.
3. By actually letting the main application make the actual call.

Problem with #1 and #2 is, if API of underlying application changes; my tests would keep passing and code will actual break and hence defeating the purpose of tests in first place.

Problem with #3 is, there is no way I can do rollback of data, the way test suite does on teardown. And I am running my tests parallely and hence if I let actual web services go, I will get errors such as "username taken" or stuff.

So the question to community is, what is the best practice?

Answer 1

I can't see that there is a middleground between isolating your client from the service and actually hitting the service. You could have erroneously passing tests because the service has changed behavior, but don't you have some "contract" with the development team working on that service that holds them responsible for breakage?

You might try fakeweb and get a fresh copy of expected results each day so your tests won't run against stale data responses.

Answer 2

Put your main application in a development or staging environment. Spin up your web service in the same environment. Have the one call the other. Control the fixture data for both. That's not mocking; you're not getting a fake implementation.

Not only will this allow you to be more confident in real-world stability, it will allow you to test performance in a staging environment, and also allow you to test your main application against a variety of versions of the web service. It's important that your tests don't do the wrong thing when your web service changes, but it's even more important that your main application doesn't either. You really want to know this confidently before either component is upgraded in production.