A simple freeze behavior decorator

Question

Hi,

I'm trying to write a freeze decorator for Python.

The idea is as follows :

(In response to the two comments)

I might be wrong but I think there is two main use of test case.

• One is the test-driven development : Ideally , developers are writing case before writing the code. It usually helps defining the architecture because this discipline forces to define the real interfaces before development. One may even consider that in some case the person who dispatches job between dev is writing the test case and use it to illustrate efficiently the specification he has in mind. I don't have any experience of the use of test case like that.

• The second is the idea that all project with a decent size and a several programmers is suffering from broken code. Something that use to work may get broken from a change that looked like an innocent refactoring. Though good architecture, loose couple between component may help to fight against this phenomenon ; you will sleep better at night if you have written some test case to make sure that nothing will break your program's behavior.

HOWEVER, Nobody can deny the overhead of writting test cases. In the first case one may argue that test case is actually guiding development and is therefore not to be considered as an overhead.

Frankly speaking, I'm a pretty young programmer and if I were you, my word on this subject is not really valuable... Anyway, I think that mosts company/projects are not working like that, and that unit tests are mainly used in the second case...

In other words, rather than ensuring that the program is working correctly, it is aiming at checking that it will work the same in the future.

This needs can be met without the cost of writing tests, by using this freezing decorator.

Let's say you have a function

def pow(n,k):
    if n == 0:  return 1
    else:       return n * pow(n,k-1)

It is perfectly nice, and you want to rewrite it as an optimized version. It is part of a big project. You want it to give back the same result for a few value. Rather than going through the pain of test cases, one could use some kind of freeze decorator.

Something such that the first time the decorator is run, the decorator run the function with the defined args (below 0, and 7) and saves the result in a map ( f --> args --> result )

@freeze(2,0)
@freeze(1,3)
@freeze(3,5)
@freeze(0,0)
def pow(n,k):
    if n == 0:  return 1
    else:       return n * pow(n,k-1)

Next time the program is executed, the decorator will load this map and check that the result of this function for these args as not changed.

I already wrote quickly the decorator (see below), but hurt a few problems about which I need your advise...

from __future__ import with_statement
from collections import defaultdict
from types import GeneratorType
import cPickle

def __id_from_function(f):
    return ".".join([f.__module__, f.__name__])

def generator_firsts(g, N=100):
    try:
        if N==0:
            return []
        else:
            return  [g.next()] + generator_firsts(g, N-1)
    except StopIteration :
        return []

def __post_process(v):
    specialized_postprocess = [
        (GeneratorType, generator_firsts),
        (Exception,     str),
    ]
    try:
        val_mro = v.__class__.mro()
        for ( ancestor, specialized ) in specialized_postprocess:
            if ancestor in val_mro:
                return specialized(v)
        raise ""
    except:
        print "Cannot accept this as a value"
        return None

def __eval_function(f):
    def aux(args, kargs):
        try:
            return ( True, __post_process( f(*args, **kargs) ) )
        except Exception, e:
            return ( False, __post_process(e) )
    return aux

def __compare_behavior(f, past_records):
    for (args, kargs, result) in past_records:
        assert __eval_function(f)(args,kargs) == result

def __record_behavior(f, past_records, args, kargs):
    registered_args = [ (a, k) for (a, k, r) in past_records ]
    if (args, kargs) not  in registered_args:
        res = __eval_function(f)(args, kargs)
        past_records.append( (args, kargs, res) )

def __open_frz():
    try:
        with open(".frz", "r") as __open_frz:
            return cPickle.load(__open_frz)
    except:
        return defaultdict(list)

def __save_frz(past_records):
    with open(".frz", "w") as __open_frz:
        return cPickle.dump(past_records, __open_frz)


def freeze_behavior(*args, **kvargs):
    def freeze_decorator(f):
        past_records = __open_frz()
        f_id = __id_from_function(f)
        f_past_records = past_records[f_id]
        __compare_behavior(f, f_past_records)
        __record_behavior(f, f_past_records, args, kvargs)
        __save_frz(past_records)
        return f
    return freeze_decorator

• Dumping and Comparing of results is not trivial for all type. Right now I'm thinking about using a function (I call it postprocess here), to solve this problem. Basically instead of storing res I store postprocess(res) and I compare postprocess(res1)==postprocess(res2), instead of comparing res1 res2. It is important to let the user overload the predefined postprocess function. My first question is : Do you know a way to check if an object is dumpable or not?

• Defining a key for the function decorated is a pain. In the following snippets I am using the function module and its name. ** Can you think of a smarter way to do that. **

• The snippets below is kind of working, but opens and close the file when testing and when recording. This is just a stupid prototype... but do you know a nice way to open the file, process the decorator for all function, close the file...

• I intend to add some functionalities to this. For instance, add the possibity to define an iterable to browse a set of argument, record arguments from real use, etc. Why would you expect from such a decorator?

• In general, would you use such a feature, knowing its limitation... Especially when trying to use it with POO?

Answer 1

Are you looking to implement invariants or post conditions?

You should specify the result explicitly, this wil remove most of you problems.

Answer 2

"In general, would you use such a feature, knowing its limitation...?"

Frankly speaking -- never.

There are no circumstances under which I would "freeze" results of a function in this way.

The use case appears to be based on two wrong ideas: (1) that unit testing is either hard or complex or expensive; and (2) it could be simpler to write the code, "freeze" the results and somehow use the frozen results for refactoring. This isn't helpful. Indeed, the very real possibility of freezing wrong answers makes this a bad idea.

First, on "consistency vs. correctness". This is easier to preserve with a simple mapping than with a complex set of decorators.

Do this instead of writing a freeze decorator.

print "frozen_f=", dict( (i,f(i)) for i in range(100) )

The dictionary object that's created will work perfectly as a frozen result set. No decorator. No complexity to speak of.

Second, on "unit testing".

The point of a unit test is not to "freeze" some random results. The point of a unit test is to compare real results with results developed another (simpler, more obvious, poorly-performing way). Usually unit tests compare hand-developed results. Other times unit tests use obvious but horribly slow algorithms to produce a few key results.

The point of having test data around is not that it's a "frozen" result. The point of having test data is that it is an independent result. Done differently -- sometimes by different people -- that confirms that the function works.

Sorry. This appears to me to be a bad idea; it looks like it subverts the intent of unit testing.

---

"HOWEVER, Nobody can deny the overhead of writting test cases"

Actually, many folks would deny the "overhead". It isn't "overhead" in the sense of wasted time and effort. For some of us, unittests are essential. Without them, the code may work, but only by accident. With them, we have ample evidence that it actually works; and the specific cases for which it works.