How to make a thread try to reconnect to the Database x times using JDBCTemplate

Question

Hi,

I have a single thread trying to connect to a database using JDBCTemplate as follows:

JDBCTemplate jdbcTemplate =  new JdbcTemplate(dataSource); 

try{
    jdbcTemplate.execute(new CallableStatementCreator() {
        @Override
        public CallableStatement createCallableStatement(Connection con)
        throws SQLException {
            return con.prepareCall(query);
        }
    }, new CallableStatementCallback() {
        @Override
        public Object doInCallableStatement(CallableStatement cs)
        throws SQLException {
            cs.setString(1, subscriberID);
            cs.execute();
            return null;
        }
    });
 } catch (DataAccessException dae) {
     throw new CougarFrameworkException(
             "Problem removing subscriber from events queue: "
             + subscriberID, dae);
 }

I want to make sure that if the above code throws DataAccessException or SQLException, the thread waits a few seconds and tries to re-connect, say 5 more times and then gives up. How can I achieve this? Also, if during execution the database goes down and comes up again, how can i ensure that my program recovers from this and continues running instead of throwing an exception and exiting?

Thanks in advance.

Answer 1

something like this:

private int retries;

/**
 * Make this configurable.
 */
public void setRetries(final int retries) {
    Assert.isTrue(retries > 0);
    this.retries = retries;

}

public Object yourMethod() {

    final int tries = 0;
    Exception lastException = null;
    for (int i = 0; i < this.retries; i++) {
        try {

            return jdbcTemplate.execute ... (your code here);

        } catch (final SQLException e) {
            lastException = e;
        } catch (final DataAccessException e) {
            lastException = e;
        }
    }
    throw lastException;

}

Answer 2

Try this. My considerations are : run a loop until the statements executed successfully. If there is a failure, tolerate the failure for 5 times and each time it will wait for 2 seconds for the next execution.

JDBCTemplate jdbcTemplate =  new JdbcTemplate(dataSource); 
boolean successfullyExecuted = false;
int failCount = 0;
while (!successfullyExecuted){
 try{
    jdbcTemplate.execute(new CallableStatementCreator() {
        @Override
        public CallableStatement createCallableStatement(Connection con)
        throws SQLException {
            return con.prepareCall(query);
        }
    }, new CallableStatementCallback() {
        @Override
        public Object doInCallableStatement(CallableStatement cs)
        throws SQLException {
            cs.setString(1, subscriberID);
            cs.execute();
            return null;
        }
    });
    successfullyExecuted = true;
 } catch (DataAccessException dae) {
     if (failedCount < 5){
        failedCount ++;
        try{java.lang.Thread.sleep(2 * 1000L); // Wait for 2 seconds
        }catch(java.lang.Exception e){}
     }else{
     throw new CougarFrameworkException(
             "Problem removing subscriber from events queue: "
             + subscriberID, dae);
     }
 } catch (java.sql.SQLException sqle){
     if (failedCount < 5){
        failedCount ++;
     }else{
     try{java.lang.Thread.sleep(2 * 1000L); // Wait for 2 seconds
     }catch(java.lang.Exception e){}
     throw new CougarFrameworkException(
             "Problem removing subscriber from events queue: "
             + subscriberID, dae);
     }
 }
}

Answer 3

It might be worthwhile for you to look into Spring's Aspect support. What you're describing is retry with (constant) backoff, and chances are you'll eventually need it somewhere else, be it talking to a web service, an email server, or any other complicated system susceptible to transient failures.

For instance, this simple method invokes the underlying method up to maxAttempts times whenever an exception is thrown, unless it is a subclass of a Throwable listed in noRetryFor.

private Object doRetryWithExponentialBackoff(ProceedingJoinPoint pjp, int maxAttempts,
        Class<? extends Throwable>[] noRetryFor) throws Throwable {
    Throwable lastThrowable = null;

    for (int attempts = 0; attempts < maxAttempts; attempts++) {
        try {
            pauseExponentially(attempts, lastThrowable);
            return pjp.proceed();
        } catch (Throwable t) {
            lastThrowable = t;

            for (Class<? extends Throwable> noRetryThrowable : noRetryFor) {
                if (noRetryThrowable.isAssignableFrom(t.getClass())) {
                    throw t;
                }
            }
        }
    }

    throw lastThrowable;
}


private void pauseExponentially(int attempts, Throwable lastThrowable) {
    if (attempts == 0)
        return;

    long delay = (long) (Math.random() * (Math.pow(4, attempts) * 100L));
    log.warn("Retriable error detected, will retry in " + delay + "ms, attempts thus far: "
            + attempts, lastThrowable);

    try {
        Thread.sleep(delay);
    } catch (InterruptedException e) {
        // Nothing we need to do here
    }
}

This advice could be applied to any bean you wish using Spring's Aspect support. See http://static.springsource.org/spring/docs/2.5.x/reference/aop.html for more details.