If transactions over REST are unachievable, how can REST ever be really useful?

Question

When looking into REST one of the first things probably anybody will notice is there isn't any transaction semantics defined, some say this is implicitly against what REST is, whilst others say any attempt to do so would result in 'tainting' the REST systems.

But lets say for arguments sake that REST did become a popular 'api' choice, and every site in the universe started to expose restful entry points.

How exactly are these usable without transaction behavior (and I'm talking non compensating)? because it seems to me one of the benefits of REST is that it breaks down the components of data, this you would think opens them up to having smart clients compose data (and add and adjust such composed data) from multiple services. But if I cannot do my changes to this composition of data atomically and in isolation then using REST becomes useless.

As time goes on and the need for serious data exposition arrives, we're gonna want something that is: Simple (REST wins there), and supports transactional behavior so we can manipulate this data reliably.

Now, I've read a specific argument a couple of times in my research, and its related to how we're supposed to think about transactions in REST, and the example given is the shopping cart, where you implicitly have isolation because the cart is YOURS.

However I disagree with this argument, firstly, the isolation a shopping cart has is merely convenient, this isn't a transaction isolation.. what happens if I'm simultaneously doing an operation against my cart whilst some part of my application is reading data from it? I wouldn't expect the reading part of my application to see data that is 'still in transaction'.

Not to mention the fact that not all data changes have an implicit transaction model, transactions over multiple services definitely don't.

It seems to me that transactions need to happen, and need to happen in a way that enables the actual REST calls to be ignorant of the fact (adding to the rest payload being a big no no, but adding headers being OK).

I've read a few suggestions about how a transaction model can be created over REST, and some of the specifications being written seem to be very recent.

Are there any real thoughts about this? shouldn't there be something 'more' than REST so that REST's simplistic nature can be harnessed against solid data manipulation ('acid' transactions).

If not are we expected to really UP the ante, and tell service developers that if they want to interact in a pure data world they need to support something arguably monolithic like soap? or even worse try and build their own custom transaction support into something like REST, making each service non-standard and breaking the entire power of REST?

Thanks in advance for any thoughts.

---

Edit, added brief scenario:

Imagine a client form that handle creation of an Album, for convenience on that Album, rather than asking the user to give the uri for the artist resource, they can pick from a list of artists (most likely GET'd from the artists catalog).

For sake of usability the client can write the artist name in manually so they can create an artist 'inline'.. in the posting scenario, the client code understands this, and before sending the request to create the album, it firstly attempts to determine if the artist already exists, if so, gets the uri for that artist, otherwise creates the artist and gets the artists uri.

The client code then continues on to create the album, this is the smarter than usual client, it isn't sat right on top of REST and 'dumbly' posting, but instead has some interaction that handles the purer REST logic.

However in this scenario it would be nice to guarantee that the artist isn't created unless the album is, given the artist is created first.

This isn't as 'critical' as a transaction would imply, but it defines a set of work the client code would prefer to be happening as one operation (after all, it IS making this look like a single operation to the user).

The only guidance I've seen for this scenario is to have the client do compensating action in the event the album creation fails, and specifically call to delete the artist. But this seems problematic, because the client is assuming the artist was isolated, as unlikely as it may be, what happens if another client already 'saw' that artist, and assigns to it?

These are my concerns regarding doing data changes, and whilst there are certainly other gaps (who says the artist couldn't be just deleted at a later date anyway), those actions are NOT transparent (ie, the actions aren't something automated by the client, but something a user has specifically requested).

I hope that helps illuminate the topic some.

Answer 1

Take a look at the OpenStreetMap API. I think it is kind of "REST" and it provides kind of "transactions" – called 'changesets' there. Is that what you need?

Answer 2

I know of very successful online accounting systems (SaaS) which have a REST-based non-transactional API for creating, retrieving and manipulating invoices etc. They are widely acclaimed for their API and the ease of integration their system with other parties. The API is easy to maintain, usable from a variety of platforms and ensuring backwards compatability is reasonably straightforward.

The lack of transactions can be a real pain in the neck, but most of the time, when their servers aren't too loaded, the API works very well.

Sometimes, less-than-perfect is just good enough.

Answer 3

This is an interesting topic. As you mentioned, SOAP has this sort of functionality already, but it took many years before SOAP matured to the point where people would consider doing real security and transactions with it. Before that, it was CORBA.

The various high-grade extensions to SOAP, including security and transactions, took a lot of work by a lot of people to get them right. This isn't going to happen to REST overnight, and it may never happen at all.

I feel that much of REST's currently popularity is something of a backlash against poorly-designed and over-complicated SOAP implementations, which is a shame, because SOAP doesn't have to be like that. As with anything else, it needs good design to make it work nicely.

Answer 4

A REST operation can start a transaction, perform multiple database or other transactional operations, then commit or rollback - all within the scope of a transaction.

What REST cannot do is to be other than the root of a transaction. You can't start a transaction, then perform two REST operations and a database operation, then commit them all together. That's just like the situation of ASMX web services in .NET. They can be the root of a transaction, but that's all. They were successful for years, until WCF was introduced, supporting WS-Transactions. Even today and using WCF, most web service operations do not need to be transactional in the sense you're asking about.

Answer 5

If you want transactions in a ReST application, at the ReST API function, it's usually because you still have your techie webservice guy googles on.

Let's look at it another way.

Soap googles on: Open a transaction, create a customer record, create an order record, commit transaction.

ReST googles on: Ask server what to do. Server says "create customer resource", so POST /customers. Server says, "now create an order if you want", client creates the order by following the form.

In ReST, the application protocol is expressed in terms of resources being created and manipulated, not in terms of data that has transaction boundaries.

If you want to have a long-running transaction that spans all those operations, it's the server that decides to initiate it, not the client.

You can still implement long running transactions on the server side. If you attempt to want transactions from the client side, then you assume the client already knows all the operations it's going ot execute and the transaction boundaries that exist between those operations. If that's what your expectations are of the client, you've already given up the late binding, hypermedia-driven nature of a rest architecture.

So indeed, if you're not doing ReST and are trying to fit in RPC over http, you'll have a problem with not having transactions.

Answer 6

I am going to assume that when you talk about transactions you are talking about a distributed Two Phase Commit protocol.

If I understand correctly you are trying to understand how we could ever use REST to perform operations that span multiple systems if REST cannot support transactions across distinct REST requests. The problem is you are making a potentially flawed assumption that we should be using transactions to achieve consistency. What price are we paying for using them, and what alternatives exist?

Pat Helland who used to work for Amazon and is now at Microsoft, wrote a paper Life beyond Distributed Transactions. In the paper makes the following statement:

Unfortunately, programmers striving to solve business goals like eCommerce, supply-chain-management, financial, and health-care applications increasingly need to think about scaling without distributed transactions. They do this because attempts to use distributed transactions are too fragile and perform poorly.

His paper explores alternative solutions to distributed transactions that do scale and perform well.

Maybe REST will be successful because it does not support transactions. Here is a quote from Roy Fielding, the guy who invented the term REST

If you find yourself in need of a distributed transaction protocol, then how can you possibly say that your architecture is based on REST? I simply cannot see how you can get from one situation (of using RESTful application state on the client and hypermedia to determine all state transitions) to the next situation of needing distributed agreement of transaction semantics wherein the client has to tell the server how to manage its own resources.

...for now I consider "rest transaction" to be an oxymoron.

This is from a message on the REST-discuss list from June 9th, 2009. I can't provide a link because Yahoo groups is useless.

Answer 7

I think most actions that normally require transactions can be reworked to occur without them.

For example, the classic bank transfer. Suppose I want to move $100 from account A to B:

Begin Transaction
  /Debit  A, $100  
  /Credit B, $100
Commit Transaction

This could be reworked as:

 /Transfer  A, B, $100  

In this way, the server might do this in two steps, but the action from the client is a single, atomic operation that makes logical sense.

I'm sure there are lots of examples where it is more convenient to do an all or nothing set of operations (and I'm curious what people can come up with to address them), but I usually rework things in this way.

Answer 8

Actually, transactions in REST work in much the same way that transactions are supposed to work in traditional SOAP services.

RPC-style transactions, where the client issues a "begin transaction" command (or some operation that implicitly begins but does not commit a transaction), followed by some additional operations, followed by another implicit/explicit "end transaction" command, are antiquated. The Web Service world moved away from the RPC model a long time ago; RPC/encoded in SOAP is not even WS-I compliant!

Instead of procedures, SOAP services today are built around the concept of messages. A single message contains all of the information necessary to perform a complete transaction. If you are submitting an order, the OrderRequest message will contain the customer information, order information, order details, payment details... everything that the server could possibly need to know about a single order.

We do have WS-Transactions that define semantics for distributed transactions, but these aren't really intended to support RPC-like semantics from the client, they're meant for orchestrations when you need to have several services participate in the same transaction. For example your order service needs to enlist a payment processing service to validate the credit card info and post a payment, which needs to be rolled back if the fulfillment service finds that you're out of stock, or something like that... you get the idea. This all takes place in the server environment so there's really nothing stopping you from doing the same thing in REST.

In REST, you have resources instead of messages, but the concept is really quite similar. Instead of an OrderRequest message the client submits (PUT) an Order resource, and the resource should contain all the same information necessary to complete the transaction. Admittedly, if you're trying to perform complex orchestrations, you might find REST a bit unwieldy compared to SOAP, but that doesn't mean you can't keep using REST for your front-end, just that SOAP will serve you better for your back-end services.

The reality, though, is that 99% of the time, people don't need the complexity of WS-Transactions. I know this because I use SOAP (WCF) almost exclusively and rarely use WS-Transactions.

In REST, the "state" resides on the client. The server tells the client, "here is all the information you need to give me to create this resource (complete this transaction)." But this blank form doesn't actually begin a transaction. It's up to the client to actually provide the information - fill in the form, so to speak. What the client does while it is filling in the form is of no concern to the server; when the client finally sends the finished form to the server, it is validated in its entirety. It's similar to a Unit of Work, except that the client is the one keeping track of the "work".

Hope this gives a better idea of how transactions can be and are achieved over REST. They just rely on a richer message/resource concept and a form of optimistic concurrency.

Answer 9

Because not every system requires transactions.

Answer 10

I think that transactions using REST are actually achievable. Think of a transaction as a resource which you can create (start), edit (post changes through), and delete (commit, roll back). Changes posted to the transaction wouldn't need to modify the global state until the transaction was committed, and during the commit, you could enforce any global state consistency rules you'd need to. The transaction resource would, of course, be protected through authorization rules.

You already mentioned the common illustration for this concept:

Now, I've read a specific argument a couple of times in my research, and its related to how we're supposed to think about transactions in REST, and the example given is the shopping cart, where you implicitly have isolation because the cart is YOURS.

However I disagree with this argument, firstly, the isolation a shopping cart has is merely convenient, this isn't a transaction isolation.. what happens if I'm simultaneously doing an operation against my cart whilst some part of my application is reading data from it? I wouldn't expect the reading part of my application to see data that is 'still in transaction'.

Why wouldn't you allow the viewing of the transaction? As long as you present it as what it is, a list of pending changes, it's actually helpful to provide that feature. But if you didn't want it to be viewable, you could disabling GET on the resource.