cleaning up when using exceptions and files in python

Question

Hello all,

I'm learning python for a couple of days now and am struggling with its 'spirit'. I'm comming from the C/C++/Java/Perl school and I understand that python is not C (at all) that's why I'm trying to understand the spirit to get the most out of it (and so far it's hard)...

My question is especially focused on exception handling and cleaning: The code at the end of this post is meant to simulate a fairly common case of file opening/parsing where you need to close the file in case of an error...

Most samples I have seen use the 'else' clause of a try statement to close the file... which made sense to me until I realized that the error might be due to

• the opening itself (in which case there is no need to close the not opened file)
• the parsing (in which case the file needs to be closed)

The trap here is that if you use the 'else' clause of a try bloc then the file never gets closed if the error happens during parsing! On the other end using the 'finally' clause result in an extra necessary check because the file_desc variable may not exist if the error happened during the opened (see comments in the code below)...

This extra check is inefficient and full of shit because any reasonable program may contain hundreds of symbols and parsing the results of dir() is a pain in the ass... Not to mention the lack of readability of such a statement...

Most other languages allow for variable definitions which could save the day here... but in python, everything seems to be implicit...

Normally, one would just declare a file_desc variable, then use many try/catch blocs for every task... one for opening, one for parsing and the last one for the closing()... no need to nest them... here I don't know a way to declare the variable... so I'm stuck right at the begining of the problem !

so what is the spirit of python here ???

• split the opening/parsing in two different methods ? How ?
• use some kind of nested try/except clauses ??? How ?
• maybe there is a way to declare the file_desc variable and then there would be no need for the extra checking... is it at all possible ??? desirable ???
• what about the close() statement ??? what if it raises an error ?

thanx for your hints... here is the sample code:

class FormatError(Exception):
    def __init__(self, message):
        self.strerror = message
    def __str__(self):
        return repr(message)


file_name = raw_input("Input a filename please: ")
try:
    file_desc = open(file_name, 'r')
    # read the file...
    while True:
        current_line = file_desc.readline()
        if not current_line: break
        print current_line.rstrip("\n")
    # lets simulate some parsing error...
    raise FormatError("oops... the file format is wrong...")
except FormatError as format_error:
    print "The file {0} is invalid: {1}".format(file_name, format_error.strerror)
except IOError as io_error:
    print "The file {0} could not be read: {1}".format(file_name, io_error.strerror)
else:
    file_desc.close()
# finally:
#     if 'file_desc' in dir() and not file_desc.closed:
#        file_desc.close()

if 'file_desc' in dir():
    print "The file exists and closed={0}".format(file_desc.closed)
else:
    print "The file has never been defined..."

Answer 1

The easiest way to deal with this is to use the fact that file objects in Python 2.5+ are context managers. You can use the with statement to enter a context; the context manager's __exit__ method is automatically called when exiting this with scope. The file object's context management automatically closes the file then.

try:
    with file("hello.txt") as input_file:
        for line in input_file:
            if "hello" not in line:
                 raise ValueError("Every line must contain 'hello'!")
except IOError:
    print "Damnit, couldn't open the file."
except:
    raise
else:
    print "Everything went fine!"

The open hello.txt handle will automatically be closed, and exceptions from within the with scope are propagated outside.

Answer 2

As of Python 2.5, there's a with command that simplifies some of what you're fighting with. Read more about it here. Here's a transformed version of your code:

class FormatError(Exception):
    def __init__(self, message):
        self.strerror = message
    def __str__(self):
        return repr(message)


file_name = raw_input("Input a filename please: ")
with open(file_name, 'r') as file_desc:
    try:
        # read the file...
        while True:
            current_line = file_desc.readline()
            if not current_line: break
            print current_line.rstrip("\n")
        # lets simulate some parsing error...
        raise FormatError("oops... the file format is wrong...")
    except FormatError as format_error:
        print "The file {0} is invalid: {1}".format(file_name, format_error.strerror)
    except IOError as io_error:
        print "The file {0} could not be read: {1}".format(file_name, io_error.strerror)

if 'file_desc' in dir():
    print "The file exists and closed={0}".format(file_desc.closed)
else:
    print "The file has never been defined..."

Answer 3

OK, I'm an ass. edit:and BTW, many thanx for those who already answered while I was posting this

The code below does the trick. You must create a nested block with the 'with as' statement to make sure the file is cleaned:

class FormatError(Exception):
    def __init__(self, message):
        self.strerror = message
    def __str__(self):
        return repr(message)


file_name = raw_input("Input a filename please: ")
try:
    #
    # THIS IS PYTHON'S SPIRIT... no else/finally
    #
    with open(file_name, 'r') as file_desc:
        # read the file...
        while True:
            current_line = file_desc.readline()
            if not current_line: break
            print current_line.rstrip("\n")
        raise FormatError("oops... the file format is wrong...")
    print "will never get here"
except FormatError as format_error:
    print "The file {0} is invalid: {1}".format(file_name, format_error.strerror)
except IOError as io_error:
    print "The file {0} could not be read: {1}".format(file_name, io_error.strerror)

if 'file_desc' in dir():
    print "The file exists and closed={0}".format(file_desc.closed)
else:
    print "The file has never been defined..."

Answer 4

Close can to my knowledge never return an error.

In fact, the file handle will be closed when garbage collected, so you don't have to do it explicitly in Python. Although it's still good programming to do so, obviously.

Answer 5

Just a note: you can always declare a variable, and then it would become something like this:

file_desc = None
try:
    file_desc = open(file_name, 'r')
except IOError, err:
    pass
finally:
    if file_desc:
        close(file_desc)

Of course, if you are using a newer version of Python, the construct using context manager is way better; however, I wanted to point out how you can generically deal with exceptions and variable scope in Python.