Design/Architecture question: rollbacks with remote services

Question

Hi,

For example, there's remote API with the following calls:

getGroupCapacity(group)
setGroupCapacity(group, quantity)
getNumberOfItemsInGroup(group)
addItemToGroup(group, item)
deleteItemFromGroup(group, item)

The task is to add some item to some group. Groups have capacity. So first we should check if group is not full. If it is, increase capacity, then add item. Something like this (for example API is exposed with SOAP):

function add_item($group, $item) {
   $soap = new SoapClient(...);
   $capacity = $soap->getGroupCapacity($group);
   $itemsInGroup = $soap->getNumberOfItemsInGroup($group);
   if ($itemsInGroup == $capacity) {
       $soap->setGroupCapacity($group, $capacity + 1);
   }
   $soap->addItemToGroup($group, $item);
}

Now what if addItemToGroup failed (item was bad)? We need to rollback group's capacity.

Now imagine that you have to add 10 items to group and then setup added items with some properties - and all this in a single transaction. That means if it fails somewhere in the middle you must rollback everything to previous state.

Is it possible without bunch of IF's and spaghetti code? Any library, framework, pattern, or architecture decision which will simplify such operations (in PHP)?

UPD: SOAP is just an example. Solution should fit any service, even raw TCP. The main point of the question is how to organize transactional behavior with underlying non-transactional API.

UPD2: I guess this problem is pretty same in all programming languages. So any answers are welcomed, not only PHP.

Thanks in advance!

Answer 1

PHP Exceptions

You could encapsulate the individual SOAP queries in classes throwing appropriate exceptions.

A dirtier solution would be to create an exception array and manually add queryStatus = false or queryStatus = true to each step and then check if the proposed transaction is valid. If so, you call a final commitTransaction method.

Answer 2

Theoretically, one of "WS-DeathStar"-protocol family, namely WS-Transaction deals precisely with that. However, I'm not aware (I'm not a PHP dev, though) of any implementations of this standard in PHP.

Answer 3

Remote services generally do not support transactions. I do not know PHP, but in BPEL you have something called as Compensation.

Compensation, or undoing steps in the business process that have already completed successfully, is one of the most important concepts in business processes. The goal of compensation is to reverse the effects of previous activities that have been carried out as part of a business process that is being abandoned.

Perhaps you could try something similar. There'll be some if/else.

Answer 4

<?php
//
// Obviously better if the service supports transactions but here's
// one possible solution using the Command pattern.
//
// tl;dr: Wrap all destructive API calls in IApiCommand objects and
// run them via an ApiTransaction instance.  The IApiCommand object
// provides a method to roll the command back.  You needn't wrap the
// non-destructive commands as there's no rolling those back anyway.
//
// There is one major outstanding issue: What do you want to do when
// an API command fails during a rollback? I've marked those areas
// with XXX.
//
// Barely tested but the idea is hopefully useful.
//

class ApiCommandFailedException extends Exception {}
class ApiCommandRollbackFailedException extends Exception {}
class ApiTransactionRollbackFailedException extends Exception {}

interface IApiCommand {
    public function execute();
    public function rollback();
}


// this tracks a history of executed commands and allows rollback    
class ApiTransaction {
    private $commandStack = array();

    public function execute(IApiCommand $command) {
        echo "EXECUTING " . get_class($command) . "\n";
        $result = $command->execute();
        $this->commandStack[] = $command;
        return $result;
    }

    public function rollback() {
        while ($command = array_pop($this->commandStack)) {
            try {
                echo "ROLLING BACK " . get_class($command) . "\n";
                $command->rollback();
            } catch (ApiCommandRollbackFailedException $rfe) {
                throw new ApiTransactionRollbackFailedException();
            }
        }
    }
}


// this groups all the api commands required to do your
// add_item function from the original post.  it demonstrates
// a nested transaction.
class AddItemToGroupTransactionCommand implements IApiCommand {
    private $soap;
    private $group;
    private $item;
    private $transaction;

    public function __construct($soap, $group, $item) {
        $this->soap = $soap;
        $this->group = $group;
        $this->item = $item;
    }

    public function execute() {
        try {
            $this->transaction = new ApiTransaction();
            $this->transaction->execute(new EnsureGroupAvailableSpaceCommand($this->soap, $this->group, 1));
            $this->transaction->execute(new AddItemToGroupCommand($this->soap, $this->group, $this->item));
        } catch (ApiCommandFailedException $ae) {
            throw new ApiCommandFailedException();
        }
    }

    public function rollback() {
        try {
            $this->transaction->rollback();
        } catch (ApiTransactionRollbackFailedException $e) {
            // XXX: determine if it's recoverable and take
            //      appropriate action, e.g. wait and try
            //      again or log the remaining undo stack
            //      for a human to look into it.
            throw new ApiCommandRollbackFailedException();
        }
    }
}


// this wraps the setgroupcapacity api call and
// provides a method for rolling back    
class EnsureGroupAvailableSpaceCommand implements IApiCommand {
    private $soap;
    private $group;
    private $numItems;
    private $previousCapacity;

    public function __construct($soap, $group, $numItems=1) {
        $this->soap = $soap;
        $this->group = $group;
        $this->numItems = $numItems;
    }

    public function execute() {
        try {
            $capacity = $this->soap->getGroupCapacity($this->group);
            $itemsInGroup = $this->soap->getNumberOfItemsInGroup($this->group);
            $availableSpace = $capacity - $itemsInGroup;
            if ($availableSpace < $this->numItems) {
                $newCapacity = $capacity + ($this->numItems - $availableSpace);
                $this->soap->setGroupCapacity($this->group, $newCapacity);
                $this->previousCapacity = $capacity;
            }
        } catch (SoapException $e) {
            throw new ApiCommandFailedException();
        }
    }

    public function rollback() {
        try {
            if (!is_null($this->previousCapacity)) {
                $this->soap->setGroupCapacity($this->group, $this->previousCapacity);
            }
        } catch (SoapException $e) {
            throw new ApiCommandRollbackFailedException();
        }
    }
}

// this wraps the additemtogroup soap api call
// and provides a method to roll the changes back
class AddItemToGroupCommand implements IApiCommand {
    private $soap;
    private $group;
    private $item;
    private $complete = false;

    public function __construct($soap, $group, $item) {
        $this->soap = $soap;
        $this->group = $group;
        $this->item = $item;
    }

    public function execute() {
        try {
            $this->soap->addItemToGroup($this->group, $this->item);
            $this->complete = true;
        } catch (SoapException $e) {
            throw new ApiCommandFailedException();
        }
    }

    public function rollback() {
        try {
            if ($this->complete) {
                $this->soap->removeItemFromGroup($this->group, $this->item);
            }
        } catch (SoapException $e) {
            throw new ApiCommandRollbackFailedException();
        }
    }
}


// a mock of your api
class SoapException extends Exception {}
class MockSoapClient {
    private $items = array();
    private $capacities = array();

    public function addItemToGroup($group, $item) {
        if ($group == "group2" && $item == "item1") throw new SoapException();
        $this->items[$group][] = $item;
    }

    public function removeItemFromGroup($group, $item) {
        foreach ($this->items[$group] as $k => $i) {
            if ($item == $i) {
                unset($this->items[$group][$k]);
            }
        }
    }

    public function setGroupCapacity($group, $capacity) {
        $this->capacities[$group] = $capacity;
    }

    public function getGroupCapacity($group) {
        return $this->capacities[$group];
    }

    public function getNumberOfItemsInGroup($group) {
        return count($this->items[$group]);
    }
}

// nested transaction example
// mock soap client is hardcoded to fail on the third additemtogroup attempt
// to show rollback
try {
    $soap = new MockSoapClient();
    $transaction = new ApiTransaction();
    $transaction->execute(new AddItemToGroupTransactionCommand($soap, "group1", "item1")); 
    $transaction->execute(new AddItemToGroupTransactionCommand($soap, "group1", "item2"));
    $transaction->execute(new AddItemToGroupTransactionCommand($soap, "group2", "item1"));
    $transaction->execute(new AddItemToGroupTransactionCommand($soap, "group2", "item2"));
} catch (ApiCommandFailedException $e) {
    $transaction->rollback();
    // XXX: if the rollback fails, you'll need to figure out
    //      what you want to do depending on the nature of the failure.
    //      e.g. wait and try again, etc.
}

Answer 5

It sounds like you need transactions and/or locking, just like a database. your client code would read something like:

function add_item($group, $item) {
   $soap = new SoapClient(...);
   $transaction = $soap->startTransaction();
   # or: 
   #   $lock = $soap->lockGroup($group, "w");
   # strictly to prevent duplication of the rest of the code: 
   #   $transaction = $lock;
   $capacity = $soap->getGroupCapacity($transaction, $group);
   $itemsInGroup = $soap->getNumberOfItemsInGroup($transaction, $group);
   if ($itemsInGroup == $capacity) {
       $soap->setGroupCapacity($transaction, $group, $capacity + 1);
   }
   $soap->addItemToGroup($transaction, $group, $item);
   $transaction->commit();
   # or: $lock->release();
}

Of course, you'll need to handle misbehaving clients, such as those that crash before committing/releasing or those that lock too much, causing other clients to unnecessarily fail. This is possible with inactivity and maximum timeouts, and maximum number of locks per client.

Answer 6

Put the transaction logic on the remote side. setGroupCapacity() should be encapsulated in addItemToGroup(). This is internal state, something the caller should not bother about. With this you can add item by item and easily unwind that with deleteItemFromGroup().

If you must live with a low level API then the rollback relies on you tracking your flow of actions.

Answer 7

Gregor Hohpe wrote a nice summary of the various approaches for handling errors remotely:

Your Coffee Shop Doesn’t Use Two-Phase Commit

In brief:

• Write-off: do nothing, or discard the work done.
• Retry: retry the parts that failed. Easier if you design your service to be idempotent, so that it may be run repeatedly with the same input with no ill effects.
• Compensating action: provide a service with a compensating action that lets you undo the work so far.
• Transaction coordinator: the traditional two-phase commit. Theoretically ideal, hard to pull of in practice, lots of buggy middleware out there.

However, in your case, it may be that the remote API is too fine grained. Do you really need setGroupCapacity as a separate service? How about just providing addUserToGroup, and letting the service handle any necessary capacity increase internally? That way, the entire transaction could be contained in a single service call.

Your current API also opens for concurrency issues and race conditions. What if, between the call to getNumberOfItemsInGroup and setGroupCapacity, some other thread manages to add a user? Your request will fail because the other thread "stole" your capacity increase.