Exception escapes from workflow despite TryCatch activity

Question

I have a workflow inside a Windows Service that is a loop that performs work periodically. The work is done inside a TryCatch activity. The Try property is a TransactionScope activity that wraps some custom activities that read and update a database. When the transaction fails, I would expect any exception that caused this to be caught by the TryCatch. However, my workflow aborts. The workflow I have is the following:

var wf = new While(true)
{
    Body = new Sequence
    {
        Activities =
        {
            new TryCatch
            {
                Try = new TransactionScope
                {
                    IsolationLevel = IsolationLevel.ReadCommitted,
                    Body = new Sequence
                    {
                        Activities = { ..custom database activities.. }
                    },
                    AbortInstanceOnTransactionFailure = false
                },
                Catches =
                {
                    new Catch<Exception>
                    {
                        Action = new ActivityAction<Exception>
                        {
                            Argument = exception,
                            Handler = ..log error..
                        }
                    }
                }
            },
            new Delay { Duration = new InArgument<TimeSpan>(duration) }
        }
    },
}

In my case, it's possible that the database is sometimes unavailable so obviously the transaction won't commit. What happens in this case is that the workflow aborts with the following exception:

System.OperationCanceledException: An error processing the current work item has caused the workflow to abort.

The inner exception is:

System.Transactions.TransactionException: The operation is not valid for the state of the transaction.

This makes sense because I have just switched off the database. However, why isn't this exception handled by my TryCatch activity?

EDIT 1: Some additional information. I run the workflow using the WorkflowApplication class. To better see what's going on, I specified the properties Aborted and OnUnhandledException. When the exception occurs, it goes directly to Aborted and OnUnhandledException is skipped (although this is clearly an unhandled exception).

EDIT 2: I enabled the debug log and this provides some additional insight. The 'custom database activities' successfully run to completion. The first event log entry that indicates that something is wrong is a Verbose level message: The runtime transaction has completed with the state 'Aborted'. Next I see an Information message: WorkflowInstance Id: 'dbd1ba5c-2d8a-428c-970d-21215d7e06d9' E2E Activity (not sure what this means). And the Information message after that is: Activity 'System.Activities.Statements.TransactionScope', DisplayName: 'Transaction for run immediately checks', InstanceId: '389' has completed in the 'Faulted' state.

After this message, I see that each parent (including the TryCatch activity) completes in the 'Faulted' state, ending with the abortion of my workflow.

EDIT 3: To be clear, everything works as expected when an exception occurs in any of the 'custom database activities'. The exception is caught and the workflow continues. It only goes wrong when the transaction can't commit at the end of the TransactionScope. See the following stacktrace that is logged from the Aborted callback:

at System.Transactions.TransactionStateInDoubt.Rollback(InternalTransaction tx, Exception e)
at System.Transactions.Transaction.Rollback(Exception e)
at System.Activities.Runtime.ActivityExecutor.CompleteTransactionWorkItem.HandleException(Exception exception)

If you follow the calls from TransactionScope.OnCompletion(...), eventually you will arrive at the ActivityExecutor class from the stacktrace.

Answer 1

Transactions commit asynchronously and after the fact. You can't react to a failure of the transaction to commit because of a problem at the resource manager level.

As you pointed out, you can deal with exceptions that occur in your activities. If you look at the tracking records for your workflow my guess is that you would see the TryCatch activity is closed prior to the transaction abort.

Many years ago when I was a program manager in the COM+ team I studied this issue because often people want a transactional component (or workflow) as in this case to be able to react to a transaction abort.

The async nature of the resolution of the transaction means that you simply cannot react to it in the component itself. The solution is to react in the caller which can then take some action.

The design assumption is that once a transaction has aborted, nothing about state aqcuired in the transaction can be safely used - it will all be discarded because the transaction is aborted.

Answer 2

So just to add to Ron's answer. Your only option here is to add the SqlWorkflowInstanceStore and drop a Persist activity just before the TransactionScope. When the transaction aborts the whole workflow will abort but the past saved state will still be in the persistence database and the workflow can be restarted from this previously saved state and execute the transaction again.