Print the full traceback in python (without halting the program)

Question

I'm writing a program that parses a 10 websites, locates data files, saves the files, and then parses them to make data that can be readily used in numpy. There are TONS of errors this file encounters through bad links, poorly formed xml, missing entries, and other things I've yet to categorize. I initially made this program to handle errors like this:

try:
    do_stuff()
except:
    pass

But now I want to log errors.

try:
    do_stuff()
except Exception, err:
    print Exception, err

Note this is printing to a log file for later review. This usually prints very useless data. What I want is to print the exact same lines printed when the error triggers without the try-except intercepting the exception, but I don't want it to halt my program since it is nested in a series of for loops that I would like to see to completion. Do your thing, stackoverflow!

Answer 1

I guess you want the full traceback stack. Here are some examples: http://blog.tplus1.com/index.php/2007/09/28/the-python-logging-module-is-much-better-than-print-statements/

When in doubt, the manual is here to help: http://docs.python.org/library/logging.html

I hope this points you in the right direction.

Answer 2

You will need to put the try/except inside the most innerloop where the error may occur, i.e.

for i in something:
    for j in somethingelse:
        for k in whatever:
            try:
                something_complex(i, j, k)
            except Exception, e:
                print e
        try:
            something_less_complex(i, j)
        except Exception, e:
            print e

... and so on

In other words, you will need to wrap statements that may fail in try/except as specific as possible, in the most inner-loop as possible.

Answer 3

You want the traceback module. It will let you print stack dumps like Python normally does. In particular, the print_last function will print the last exception and a stack trace.

Answer 4

Traceback.format_exc() or sys.exc_info() will yeild more info if thats what you want.

import traceback
import sys

try:
    do_stuff()
except Exception, err:
    print traceback.format_exc()
    #or
    print sys.exc_info()[0]