Unittesting aspect-oriented features.

Question

Hi,

I'd like to know what would you propose as the best way to unit test aspect-oriented application features (well, perhaps that's not the best name, but it's the best I was able to come up with :-) ) such as logging or security?

These things are sort of omni-present in the application, so how to test them properly?

E.g. say that I'm writing a Cherrypy web server in Python. I can use a decorator to check whether the logged-in user has the permission to access a given page. But then I'd need to write a test for every page to see whether it works oK (or more like to see that I had not forgotten to check security perms for that page).

This could maybe (emphasis on maybe) be bearable if logging and/or security were implemented during the web server "normal business implementation". However, security and logging usually tend to be added to the app as an afterthough (or maybe that's just my experience, I'm usually given a server and then asked to implement security model :-) ).

Any thoughts on this are very welcome. I have currently 'solved' this issue by, well - not testing this at all. Thanks.

Answer 1

Well... let's see. In my opinion you are testing three different things here (sorry for the "Java AOP jargon"):

1. the features implemented by the interceptors (i.e. the methods that implement the functions activated at the cutpoints)
2. the coverage of the filters (i.e. whether the intended cutpoints are activated correctly or not)
3. the interaction between the cutpoints and the interceptors (with the side effects)

You are unit testing (strictly speaking) if you can handle these three layers separatedly. You can actually unit test the first; you can use a coverage tool and some skeleton crew application with mock objects to test the second; but the third is not exactly unit testing, so you may have to setup a test environment, design an end-to-end test and write some scripts to input data in your application and gather the results (if it was a web app you could use Selenium, for example).

Answer 2

My answer is for the specific example you give, not for the possible problems with bolted-on security. Decorators are just regular functions, and you can test them as such. For instance:

# ... inside included module ...
def require_admin(function):
    if (admin):
        return function
    else:
        return None

@require_admin
def func1(arg1, arg2):
    pass

# ... inside unit test ...
def test_require_admin(self):
    admin = False
    f = lambda x: x
    g = require_admin(f)
    assert_equal(g, None)

    admin = True
    g = require_admin(f)
    assert_equal(g, f)

Note: this is a really terrible way to do a security check, but it gets the point across about testing decorators. That's one of the nice things about Python: it's REALLY consistent. The following expressions are equivalent:

@g
def f(x):
    return x

and

def f(x):
    return x
f = g(f)

Answer 3

IMHO, the way of testing users permissions to the pages depends on the design of your app and design of the framework you're using.

Generally, it's probably enough to cover your permission checker decorator with unit tests to make sure it always works as expected and then write a test that cycles through your 'views' (or whatever term cherrypy uses, haven't used it for a very long time) and just check if these functions are decorated with appropriate decorator.

As for logging it's not quite clear what you want test specifically. Anyway, why isn't it possible to stub the logging functionality and check what's going on there?