What is the recommended method of testing a JSON client-server api?

Question

Edit #2: Does anyone have a good method of testing the "middle" of a client-server application where we can intercept requests and responses, fake the client or server as needed, and which provides self-documentation of the api?

Cucumber might be a good solution in many cases, but it's not quite what I'm looking for. And this middle layer should be client/server implementation agnostic. (e.g., black-box).

---

Our client-server model is a ruby-on-rails server with a Flex client, using a RESTish interface with JSON as the data format. So anything the client posts to the server is usually a single JSON parameter. The server does it's thing and responds with a pure JSON model.

We have standard rails testing on the server and we're working on getting proper FlexUnit tests completed on the client (it's a moving target). However, there's a debate in my team about the effectiveness of the current testing model, since every change on the server seems to break part of the API. This is telling me that there is both a problem with API communication (between team members, self-documentation in code, etc..), and a lack of proper API sanity testing.

So I've been questioning whether we need to have a mock client for testing the server at a pure JSON level (without all the other complexities of a rich client), and possibly a mock-server for doing the same thing with the rich client. This would serve two purposes, to document the API and to provide more thorough testing of the API itself.

The reason there's a debate is that the rails guy claims that the rails integration testing is sufficient for testing all the server requests, and the middle-ground testing environment would simply be redundant.

So the question here is, given our situation, how should be go about self-documenting the API, and how should we test the API itself?

EDIT:

We have routes like /foo/444/bar.js, but the parameters can be virtually any complex JSON string depending on the action, e.g.:

json={
  "foo":{
    "x":1, 
    "y":2
   }, 
  "bar":[1,2,3,4,5]
}

but besides manually-edited API docs, there's no self-documentation. The rails controller often just deserializes and applies changes directly to the model. Would be nice to have common tests to tell us when it's changed, and what's expected.

Answer 1

You can think of it as two different projects.. if you had two project, you would've writing two separate test suites right?

You should start by establishing the API between the server and the client- as if you wont have any communication between the teams after you start implementhing.

Then you build the client that consume the API and a server that produce the API (or the tests first, if you TDD).

For testing, one team need a mock-server to supply fake API responses to test the client, and the other team need to test the produced data of the server (i.e, the second team is using ails integration testing like your rails guy claims)

Answer 2

I would recommend Cucumber. It allows you to write specific tests for your application by emulating a browser. This way you can send requests and validate the JSON response easily.

Answer 3

I just started looking at this web functional testing tool called Maxq and I think it has the potential to solve your problem, Maxq acts as a proxy server between your web client and server application. It sits on top of Junit so that means you could do proper unit testing for your API by asserting the behavior and responses of calls to your server app. It basically captures and records all the requests you make from a web client and the responses you get back from a server, it also has the ability to generate test scripts of your request which you could use to play back and test on any server.

You should try it out http://maxq.tigris.org/