Handling errors and feedback when performing bulk operations in a multi-tiered architecture

Question

Let's say you have a business logic method that can perform some operation across a number of objects. Maybe you want to call a lottery number picking web service, once for each person selected from a list. In Java, the code might look something like this:

Set<Person> selectedPeople = ... // fetch list of people
for ( Person person : selectedPeople ) {
    String lotteryNumber = callLotteryNumberWebService( person );
    // ...
}

Note that the lottery number web service may have side-effects, like recording that the person has requested a lottery number (perhaps charging their account), so even if the web service call fails for one person, it may have succeeded for others. This information (the lottery numbers) will need to be fed back to some higher level (the view).

If this were a case where a single operation took place, the business logic method could return a single value (say, the lottery number) or throw an exception with any details of the failure. But for bulk operations, it would be possible for a few of the operations to succeed and a few to fail.

This seems like a type of problem that would occur in many applications and there should be a clean way to handle it. So, what is the best way to feed this type of information back from the business logic layer to another layer in an application (like a view), preferably in a generic way that can be reused for different types of data and operations?

Answer 1

I would prefer to return a collection of custom-made error objects identifying the object, which is effected by the error, error code and description. This way the errors can be tried to remedied or displayed further to the user.

Answer 2

I'd probably return a result map of type Map<Person,Future<String>> from my getLotteryNumbers<Collection<Person>> service.

I'd then iterate through the map and use the Future.get() to get either the lottery number or the exception thrown.

In some of my services I like to implement all of the calls as single item calls and then have logic in my service to batch them up and process them as a group. The batching is implemented using a LinkedBlockingQueue and a polling thread.

In this scenario I return a Future<Thing> which waits for the batch results to be available using a CountdownLatch.

Take a look at Java Concurrency in practice to see how these components can all work together http://jcip.net/

Answer 3

I'd look into DTOs for this kind of task. The DTO could also include information on whether the persisting was succesful or not and other kinds of "metadata".

Answer 4

I think you really are overusing exceptions if you are thinking in these terms!

It is perfectly ok to return values that mean failure, rather than throwing an exception. Often it is better. Exceptions are best used when you can't recover at the level of abstraction you are at, but you should not be using them as the main means of control flow or your programs will get very hard to read.

The web service does not return exceptions, it returns return codes and messages. I would store some useful representation that presents the information returned, and return the list of those for the view or whatever is going to be looking at it.

Answer 5

This question highlights important differences between appropriate uses of exception handling, transactions, and the idea workflow "compensation" which is what the asker is trying to get at, when correctly stating:

This seems like a type of problem that would occur in many applications and there should be a clean way to handle it.

It is a common problem, first some background on the transactional approach you are currently attempting:

Data transactions were originally modeled after double-entry accounting -- a single credit and a corresponding debit had to be recorded together or not at all. As transactions get bigger than this they become increasingly problematic to implement correctly, and harder to deal with a failure. As you start to carry the idea of a single transaction across system boundaries, you are most likely approaching it wrong. It can be done, but requires complex and necessarily higher-latency transaction coordinators. At a certain scale transactions are the wrong mindset, and compensation starts to make a lot more sense.

This is where you go back and look at what the business actually does. A single large transaction is most likely not the way the business people see it. Usually they see a step completed, and depending on subsequent results, different actions to compensate may be needed. This is where the idea of a workflow and compensation comes in. Here's one introduction to those concepts

For example if you order a book from Amazon, they probably don't "lock" the record while it is in your shopping cart, or even use strict transactions to determine if the book is still in-stock when the order is confirmed. They will sell it to you anyways, and ship it when they can. If they haven't managed to get it in-stock within a few weeks they will probably send you an email telling you that they are trying to meet your needs, and you can keep waiting for them to get it in-stock, or you can cancel your order. This is called compensating, and is necessary in many real-world business processes.

Finally, there is nothing exceptional about any of this. Expect that this can happen and use normal control flow. You should not use your language's exception handling features here (good rules for when to throw an exception). Nor should you rely on tool specific (WCF?) mechanisms for seeing or handling exceptions that happen within the service implementation. Communicating faults should be a normal part of your data contract (fault contracts).

Unfortunately, by "clean way to handle it" there is no flag to set that will magically take care of it, you have to continue to decompose the problem and deal with all the resulting pieces. Hopefully these concepts will connect you with what other people have done when dealing with this issue.

Summary:

• Your problem has outgrown the concept of a transaction --> look into workflow compensation.

Good luck -

Answer 6

Another way, especially for high throughput systems, would be to use a design based on queues where a processing entity would perform operations on an object and then based on the results put the object in different queues for additional processing by other entities and then move on. This would reduce bottlenecks that would arise due to additional processing that would be required e.g. processing of order for products out of stock

Answer 7

If I understand, you have a situation where some requests can pass and some can fail. I'm not sure where you want the error to come back but you could have one of the following (or a variant, or a combination):

• A list of errors and the domain objects affected. A base domain object or something with a persistable ID might be useful for re-use. E.g. a collection of errors referring to domain objects.
• You could inject (AOP, DI) into the Person object some kind of error object/message. E.g. if (person. Errors){...}
• You could wrap the Person collection into a message with header, body, error information
• All your domain objects could include an error collection accessible via an interface; or Person supports the IHasErrors interface. You could make this generic and use a base Error object supporting warnings and validation and all manner of things.

If you are in a genuine multi-tiered (rather than layered) system then you may have a message based architecture that could easily accommodate some kind of generic error/warning/validation mechanism. SOA/Ajax systems lend themselves to this.

Happy to delve a little deeper if you have some specific questions.

Answer 8

Ideally, the call to your web service should be like this.

List<Person> selectedPeople = ... //fetch list of people
callLotteryNumberWebService(selectedPeople.ToArray );

Making a web service call for each person is costly. At the server end, you need to iterate over the list and perform the operation. The server side code can throw 2 exceptions : BulkOperationFailedException - if there is a fatal error due to db down or config file missing. Further processing not possible. BulkOperationException - this contains an array of exceptions pertaining to a person. You can persist some id to uniquely refer to each object. Your code will be like this:

List<Person> selectedPeople = ... // fetch list of people 

try{
    callLotteryNumberWebService(selectedPeople.ToArray);
}catch  (BulkOperationFailedException e) {
    SOP("Some config file missing/db down.No person records processed")
}catch(BulkOperationException e)  {
    UserDefinedExceptions us =  e.getExceptions()
    foreach(exception ein us)   {
        // read unique id to find which person object failed
    }
}

construct msg based on which personobject succeeded and which failed

Operation is deemed to be succesful when no exceptions are thrown. You can have custom error codes for failures instead of using UserDefined exceptions. Constructing the BulkOperationException in the server-side is tricky. Second you need to categorise errors thrown from server side into BulkOperationFailedException and BulkOperationException. This was how i had handled in one of my projects