Transactional Executor with Retry?

Please a bit of advice on the following:

I am using a ThreadPoolTaskExecutor to execute slow-external tasks like sending emails.

I need to improve this: 1) When the a task is passed to the exeuctor, it has to wait executing it till at least the transaction of the passing operation has finished. Example: i makes no sense to email something when the order process fails and generates an exception which occurs on commitment

2) When the task fails, some retry mechanism is used to try the task again. Example: when sending the email fails, it will be retried after 5,10 minutes and then throws an exception.

How to deal with these issues? of should I just integrate some queue that offers this functionality?..

I would say : yes use a queue in a messaging infrastructure.

Personally I would use Camel for this because I am completely smitten by Camel and would use it if I would reprogram my toaster to toast the slices golden brown at breakfast.

Since you are going to mail, it will be message based anyway, so using a message based system will already reduce the impedance mismatch.

Now things as transactions, retries and parking the message on a dead letter queue comes as standard with these things. This is nice, because you can then script your way out of trouble when a email server disaster hits, by resubmitting the messages from the dead letter queue.

Integrating an ActiveMQ or a Camel is just adding a couple of dependencies and 5-10 lines in your spring configuration.

Once it is in there it is beautiful to organize background processing, notify remote systems, automate email responses, notify sysadmins of impending doom, ... You send a message, continue what you're doing, respond to the customer, while in the background the wheels are turning.

Ok, sorry : I got carried away and got way too lyrical.

Thanks for your quick response. So bascially I don't need to start a "separate" queueing server but instead just add some jar's and it will started automatically... I mean: I like it that I only have to start Spring and not other things to and keep testing simple...

edbras 2010-09-02 14:56:33

Yep. For testing, these things are implementations of interfaces which are straightforward to mock. Sending an email is a standard component in Camel, so it requires no unit tests, only tests to verify it is properly configured.

Peter Tillemans 2010-09-02 15:15:10

Just one thing: I read that Camel is very good in routing traffic. Besides email I also want to execute code (captured in a command pattern) that will make a HTTP connection to retrieve some information that has to be put in the local DB. Is Camel also well suited for these kind of operations? So basically: make these operations asynchronize and let them retry when failure occurs...

edbras 2010-09-02 18:04:36

Reading some more :)... One more question: why Camel and ActiveMQ?

edbras 2010-09-02 20:41:42

I don't know, did more reading... and not very convinced about Camel usage... it's a bit too much of a routing platform to solve my problem which is basically to decouple an operation (run it in a separate thread) but having support for things like transaction and retry... I will have a look at ActiveMQ... But ofcourse I could be wrong ... I never used Camel... Could give a more concrete example of how to decouple an operation with Camel ?

edbras 2010-09-02 21:53:33

Why Camel and ActiveMQ : because I've been using them over the last years because they're so easy to integrate. AN example : we needed to send wafermaps to an external partner as part of a workflow. This can take a long time, so a message is sent to a queue, read by a separate module which figures out how to get the data to the partner and calls back to the workflow to move the state further. This could not be done synchronously because of transactions timeouts.

Peter Tillemans 2010-09-02 23:57:09

For just a simple background worker look at the ThreadPoolExecutor in java.util.concurrent. Here is a tutorial : http://www.javamex.com/tutorials/threads/ThreadPoolExecutor.shtml.

Peter Tillemans 2010-09-03 00:03:48

Thanks. I am familiar witht he ThreadPoolExecutor as I use it, but I want to upgrade this as I need the retry and transaction facilities that jms offers...

edbras 2010-09-03 07:39:10

I think that ActiveMQ is interesting as it offers the JMS mechanism I am looking for, but it acts as a separate server that needs to be start/stopped (from what I understand). I want to start with something smaller, like an embedded JMS "server", just like you use Camel in your existing code. Then latter I could always use a standalone separate server. It all boils down to having a kind of ThreadPoolExecutor with a transaction/retry mechanism I think... How do you experience this? Can ActiveMQ be embedded instead runing it as a standalone server ?

edbras 2010-09-03 08:04:04

ActiveMQ can be started by adding <bean id="broker" class="org.apache.activemq.xbean.BrokerFactoryBean"> <property name="config" value="classpath:org/activemq/xbean/activemq.xml" /> <property name="start" value="true" /> </bean>to the spring configuration. You do not need to define destinations, as activemq will create them for you (with default settings) the first time you use them. For more details see http://activemq.apache.org/spring-support.html

Peter Tillemans 2010-09-03 15:45:23

Thanks, I notice the usage as embedded activemq which is a good starting point. I will start of with ActiveMq and Spring-batch for my jobs. Maybe also use Camel, but don't need it (yet)..

edbras 2010-09-04 10:13:34

ansaurus

tags:

views:

answers:

Transactional Executor with Retry?

related questions