How much input validation should I be doing on my python functions/methods?

Question

I'm interested in how much up front validation people do in the Python they write.

Here are a few examples of simple functions:

def factorial(num):
    """Computes the factorial of num."""

def isPalindrome(inputStr):
    """Tests to see if inputStr is the same backwards and forwards."""

def sum(nums):
    """Same as the built-in sum()... computes the sum of all the numbers passed in."""

How thoroughly do you check the input values before beginning computation, and how do you do your checking? Do you throw some kind of proprietary exception if input is faulty (BadInputException defined in the same module, for example)? Do you just start your calculation and figure it will throw an exception at some point if bad data was passed in ("asd" to factorial, for example)?

When the passed in value is supposed to be a container do you check not only the container but all the values inside it?

What about situations like factorial, where what's passed in might be convertible to an int (e.g. a float) but you might lose precision when doing so?

Answer 1

I basically try to convert the variable to what it should be and pass up or throw the appropriate exception if that fails.

def factorial(num):
    """Computes the factorial of num."""
    try:
        num = int(num)
    except ValueError, e:
        print e
    else:
        ...

Answer 2

It rather depends on what I'm writing, and how the output gets there. Python doesn't have the public/private protections of other OO-languages. Instead there are conventions. For example, external code should only call object methods that are not prefixed by an underscore.

Therefore, if I'm writing a module, I'd validate anything that is not generated from my own code, i.e. any calls to publicly-accessible methods/functions. Sometimes, if I know the validation is expensive, I make it togglable with a kwarg:

def publicly_accessible_function(arg1, validate=False):
  if validate:
    do_validation(arg1)
   do_work

Internal methods can do validation via the assert statement, which can be disabled altogether when the code goes out of development and into production.

Answer 3

I'm trying to write docstring stating what type of parameter is expected and accepted, and I'm not checking it explicitly in my functions.

If someone wants to use my function with any other type its his responsibility to check if his type emulates one I accept well enough. Maybe your factorial can be used with some custom long-like type to obtain something you wouldn't think of? Or maybe your sum can be used to concatenate strings? Why should you disallow it by type checking? It's not C, anyway.

Answer 4

I assert what's absolutely essential.

Important: What's absolutely essential. Some people over-test things.

def factorial(num):
    assert int(num)
    assert num > 0

Isn't completely correct. long is also a legal possibility.

def factorial(num):
    assert type(num) in ( int, long )
    assert num > 0

Is better, but still not perfect. Many Python types (like rational numbers, or number-like objects) can also work in a good factorial function. It's hard to assert that an object has basic integer-like properties without being too specific and eliminating future unthought-of classes from consideration.

I never define unique exceptions for individual functions. I define a unique exception for a significant module or package. Usually, however, just an Error class or something similar. That way the application says except somelibrary.Error,e: which is about all you need to know. Fine-grained exceptions get fussy and silly.

I've never done this, but I can see places where it might be necessary.

assert all( type(i) in (int,long) for i in someList ) 

Generally, however, the ordinary Python built-in type checks work fine. They find almost all of the exceptional situations that matter almost all the time. When something isn't the right type, Python raises a TypeError that always points at the right line of code.

BTW. I only add asserts at design time if I'm absolutely certain the function will be abused. I sometimes add assertions later when I have a unit test that fails in an obscure way.

Answer 5

For calculations like sum, factorial etc, pythons built-in type checks will do fine. The calculations will end upp calling add, mul etc for the types, and if they break, they will throw the correct exception anyway. By enforcing your own checks, you may invalidate otherwise working input.

Answer 6

I almost never enforce any kind of a check, unless I think there's a possibility that someone might think they can pass some X which would produce completely crazy results.

The other time I check is when I accept several types for an argument, for example a function that takes a list, might accept an arbitrary object and just wrap it in a list (if it's not already a list). So in that case I check for the type -not to enforce anything- just because I want the function to be flexible in how it's used.

Answer 7

Only bother to check if you have a failing unit-test that forces you to.

Also consider "EAFP"... It's the Python way!