Is Implementing a HTTP, XML based CRUD Layer a good idea?

Question

I'm making a CRUD "layer" for an application. This will be a simple CRUD application that, for example, stores User information, Favorite links etc. and not do operations on "fact-type" data. It's actually only to store things like Users, Permissions, Rules, Policies etc. that other parts of the application pick up to perform work.

Overall, I want three things from this effort:

• (a) Single entry point to access CRUD functions
• (b) Ability to use any "client" to use the CRUD layer
• (c) "easy" extensibility of the CRUD where new object may be added and old objects may be changed (new fields added, nothing else removed or changed). A typical CRUD scenario?

I'm thinking I should make a Java library, expose it to clients via "REST-type-URL" (meaning just the REST-URL-way, like "users/delete/2") API via HTTP. This way, I can meet all 3 goals - CRUD layer can be on Linux, client can be in Windows.

In the CRUD layer, I will be using various things to make this happen : ORM, a web-server and other tools.

It seems the right way, but I can't help wondering that maybe this approach is too idealistic and may not work when I start implementing.

Is what I'm thinking of doing some overly-simplistic view of cramming a set of API methods into a XML fragment? (note that I'm not doing a XML-RPC, rather these XML fragments will be data only - and the XML will be sent to specific URLs such as users/update/2, which will process the XML after confirming that the XML contains information for a Users profile)

Am I right in my thinking? Does this idea have even a remote chance of working?

Any help appreciated!

Answer 1

If you are looking for a Java solution I suggest you take a look at Restlets. You are pretty much mixing XML-RPC and REST style architecture. Here are the differences between RPC and REST.
You should use the URI to determine the action and the parameters instead of passing XML if you want to do it the RESTful way.

Answer 2

For a pure data layer, have you thought about transaction support?

• Do you need to support transactions? If so, how would they be implemented?
• Will the transactions span multiple HTTP requests? (I would think for a pure REST implementation you would have multiple calls for non-trivial operations.) e.g. debit one account and credit another account.
• Who would control the transactions?

If no transaction control is required now, could you see it being required in the future? How would that affect your design?

More questions than answers. :) But food for thought.

UPDATE:

I would create a standard data access Layer using your language of choice (in your case Java) and any other libraries you need (e.g. ORM). If you already have a data access layer design, I would follow that in order to keep the application simple. If you want to expose this externally (to various clients), I would do this in what I would call a service/business layer (if the functionality is simple with no potential for reuse collapsing both to one implementation would probably be OK). In there you would handle any security (authentication, authorizations, etc.), data validation and potentially perform any mapping between your internal data representation and the external representation. You could then expose your services according to your URL based API. This gives you some separation between your actual data access implementation and your interface.

Perhaps we are talking about the same thing but our semantics just differ.

Answer 3

This is a good idea if you want access the CRUD function from multiple clients. We have been doing something like this for a while. Some implementation details that may help you.

1. Simply HTTP call is good enough, don't bother with REST. To be RESTful, you have to follow rules like (GET for READ, PUT for create etc.). Just use GET so it can be easily tested in a browser. We use XSLT to format the response into tables in HTML.

2. We use one XML schema to handle all responses. It's basically a XML representation of the SQL results, which should be flexible enough to handle multiple resultsets, rows affected, error response, return code etc.

3. Have a JSON representation of the XML so it's easier to handle in Javascript.

4. We also added a SQL backdoor so an arbitrary SQL statement can be sent to the database. This is very handy for debugging. Some security is needed for this kind of call. We only expose it to office network.

Answer 4

Yes it will work. However, you are basing your design of a distributed system on the assumption that the data model is very simple and will remain that way. If the application you are building is successful, you can be sure that new requirements will be added and new features will be requested.

Exposing the data layer as you are suggesting, tightly couples your the client applications to the data model, and your use of http makes doing multi request transactions very difficult. I know you said that you do not need to do multi-request transactions. Not today, but maybe next year?

What is the life expectancy of this application? If you say more than a couple of years then I would re-think the idea of exposing the data layer to remote clients. One of the main goals of REST is to de-couple the client and the server application to allow applications to evolve over a long period of time. If you have multiple client applications accessing a distributed service if it is not designed right, it can quickly run into nasty maintenance and versioning issues.

Edit to answer question in comments about client needing to understand model:

You have two different directions you can take regarding how the client can interact with the representation received from the server.

1. You can allow the client to have explicit knowledge of the data content contained in the representation. I.e. The client knows that there is a user name and password in certain XML elements. However, if you do this you should return a specific media type, e.g. application/vnd.mycompany.user+xml. You should not use application/xml as that does not tell the client anything about what is in the XML document. If the client were to have the knowledge that "when you go to url X" you get "an Xml document that has contains elements UserName and Password inside an element User" then you have violated the self-descriptive constraint of REST. The impact is that you have coupled the endpoint to the media-type and in effect coupled the client to that endpoint. The intent of the "hypermedia constraint" of REST is to prevent coupling between the client and the endpoints.
2. The alternative is to use a more generic media type that is simply intended to provide a client with content to display to the user and offer options to the user to allow the client to take actions. The html media type allows you to do this by providing a markup language that could be used to return the username and password in two div tags. Using the html FORM tag and A tag the client can perform additional operations on that resource. Figuring out how to make accessible all of the possible operations a "user" object might have, in a truly RESTful manner, is tricky and takes quite a bit of experience but the end result is an extremely well decoupled client and server. Look at a the web browser as an example, how often do you need to do a browser update because a web site has changed it's content.

The problem with option number two, is that using just HTML the end-user experience tends to be pretty constrained and that is where Javascript comes in. One of the "optional" REST constraints is the us of code download. Javascript is code that loaded from the server to enable additional behaviour on the client. Unfortunately, in my opinion, it also provides the ability for people to create RESTful web interfaces that return application/xml and then use the javascript to interpret that generic format. This solution works fine for the original developer of the web site that consumes the RESTful API because if the content of the xml file changes then the javascript can be changed and re-downloaded to the browser and all is good. However, for any other third party developer who is accessing this API who is not in control of the application/xml content model, their code becomes completely brittle and will potentially break if the API content changes. Ask anyone who has written any kind of client for Twitter, how many times their application has broken because Twitter changed the content.

By using the first option and giving the content a specific media type, the server developer can introduce a new media-type called application/vnd.mycompany.userV2+xml and using content negotiation the existing clients can still receive the original media type and new clients can be built to consume the new media type. The url remains the same, bookmarks are not broken, because the endpoint and the mediatype are not coupled.

If you see API documentation that provides a list of Urls and the content that is returned from those urls then chances are those developers just don't get REST and will not get the benefits that a RESTful interface provides. Ironically though, it is not those developers that will suffer the most, it is the 3rd parties attempting to interface with the API that will suffer!