Nice exception handling when re-trying code

Question

I have some test cases. The test cases rely on data which takes time to compute. To speed up testing, I've cached the data so that it doesn't have to be recomputed.

I now have foo(), which looks at the cached data. I can't tell ahead of time what it will look at, as that depends a lot on the test case.

If a test case fails cause it doesn't find the right cached data, I don't want it to fail - I want it to compute the data and then try again. I also don't know what exception in particular it will throw cause of missing data.

My code right now looks like this:

if cacheExists:
    loadCache()
    dataComputed = False
else:
    calculateData()
    dataComputed = True

try:
    foo()
except:
    if not dataComputed:
        calculateData() 
        dataComputed = True
        try:
            foo()
        except:
            #error handling code
    else:
        #the same error handling code

What's the best way to re-structure this code?

Answer 1

Is there a way to tell if you want to do foobetter() before making the call? If you get an exception it should be because something unexpected (exceptional!) happened. Don't use exceptions for flow control.

Answer 2

Using blanket exceptions isn't usually a great idea. What kind of Exception are you expecting there? Is it a KeyError, AttributeError, TypeError...

Once you've identified what type of error you're looking for you can use something like hasattr() or the in operator or many other things that will test for your condition before you have to deal with exceptions.

That way you can clean up your logic flow and save your exception handling for things that are really broken!

Answer 3

I disagree with the key suggestion in the existing answers, which basically boils down to treating exceptions in Python as you would in, say, C++ or Java -- that's NOT the preferred style in Python, where often the good old idea that "it's better to ask forgiveness than permission" (attempt an operation and deal with the exception, if any, rather than obscuring your code's main flow and incurring overhead by thorough preliminary checks). I do agree with Gabriel that a bare except is hardly ever a good idea (unless all it does is some form of logging followed by a raise to let the exception propagate). So, say you have a tuple with all the exception types that you do expect and want to handle the same way, say:

expected_exceptions = KeyError, AttributeError, TypeError

and always use except expected_exceptions: rather than bare except:.

So, with that out of the way, one slightly less-repetitious approach to your needs is:

try:
    foo1()
except expected_exceptions:
    try:
        if condition:
            foobetter()
        else:
            raise
    except expected_exceptions:
        handleError()

A different approach is to use an auxiliary function to wrap the try/except logic:

def may_raise(expected_exceptions, somefunction, *a, **k):
  try:
    return False, somefunction(*a, **k)
  except expected_exceptions:
    return True, None

Such a helper may often come in useful in several different situations, so it's pretty common to have something like this somewhere in a project's "utilities" modules. Now, for your case (no arguments, no results) you could use:

failed, _ = may_raise(expected_exceptions, foo1)
if failed and condition:
  failed, _ = may_raise(expected_exceptions, foobetter)
if failed:
  handleError()

which I would argue is more linear and therefore simpler. The only issue with this general approach is that an auxiliary function such as may_raise does not FORCE you to deal in some way or other with exceptions, so you might just forget to do so (just like the use of return codes, instead of exceptions, to indicate errors, is prone to those return values mistakenly being ignored); so, use it sparingly...!-)

Answer 4

Sometimes there's no nice way to express a flow, it's just complicated. But here's a way to call foo() in only one place, and have the error handling in only one place:

if cacheExists:
    loadCache()
    dataComputed = False
else:
    calculateData()
    dataComputed = True

while True:
    try:
        foo()
        break
    except:
        if not dataComputed:
            calculateData()
            dataComputed = True
            continue 
        else:
            #the error handling code
            break

You may not like the loop, YMMV...

Or:

if cacheExists:
    loadCache()
    dataComputed = False
else:
    calculateData()
    dataComputed = True

done = False
while !done:
    try:
        foo()
        done = True
    except:
        if not dataComputed:
            calculateData()
            dataComputed = True
            continue 
        else:
            #the error handling code
            done = True

Answer 5

I like the alternative approach proposed by Alex Martelli.

What do you think about using a list of functions as argument of the may_raise. The functions would be executed until one succeed!

Here is the code

def foo(x):
    raise Exception("Arrrgh!")
    return 0

def foobetter(x):
    print "Hello", x
    return 1

def try_many(functions, expected_exceptions, *a, **k):
    ret = None
    for f in functions:
        try:
            ret = f(*a, **k)
        except expected_exceptions, e:
            print e
        else:
            break
    return ret

print try_many((foo, foobetter), Exception, "World")

result is

Arrrgh!
Hello World
1