Is this a good approach to execute a list of operations on a data structure in Python?

Question

I have a dictionary of data, the key is the file name and the value is another dictionary of its attribute values. Now I'd like to pass this data structure to various functions, each of which runs some test on the attribute and returns True/False.

One approach would be to call each function one by one explicitly from the main code. However I can do something like this:

#MYmodule.py
class Mymodule:
  def MYfunc1(self):
  ...
  def MYfunc2(self):
  ...

#main.py
import Mymodule
...
#fill the data structure
...
#Now call all the functions in Mymodule one by one
for funcs in dir(Mymodule):
   if funcs[:2]=='MY':
      result=Mymodule.__dict__.get(funcs)(dataStructure)

The advantage of this approach is that implementation of main class needn't change when I add more logic/tests to MYmodule.

Is this a good way to solve the problem at hand? Are there better alternatives to this solution?

Answer 1

I'd say a better and much more Pythonic approach would be to define a decorator to indicate which functions you want to use:

class MyFunc(object):
    funcs = []
    def __init__(self, func):
        self.funcs.append(func)

@MyFunc
def foo():
    return 5

@MyFunc
def bar():
    return 10

def quux():
    # Not decorated, so will not be in MyFunc
    return 20

for func in MyFunc.funcs:
    print func()

Output:

5
10

Essentially you're performing the same logic: taking only functions who were defined in a particular manner and applying them to a specific set of data.

Answer 2

Sridhar, the method you proposed is very similar to the one used in the unittest module.

For example, this is how unittest.TestLoader finds the names of all the test methods to run (lifted from /usr/lib/python2.6/unittest.py):

def getTestCaseNames(self, testCaseClass):
    """Return a sorted sequence of method names found within testCaseClass
    """
    def isTestMethod(attrname, testCaseClass=testCaseClass, prefix=self.testMethodPrefix):
        return attrname.startswith(prefix) and hasattr(getattr(testCaseClass, attrname), '__call__')
    testFnNames = filter(isTestMethod, dir(testCaseClass))
    if self.sortTestMethodsUsing:
        testFnNames.sort(key=_CmpToKey(self.sortTestMethodsUsing))
    return testFnNames

Just like your proposal, unittest uses dir to list all the attributes of testCaseClass, and filters the list for those whose name startswith prefix (which is set elsewhere to equal 'test').

I suggest a few minor changes:

If you place the functions in MYmodule.py, then (of course) the import statement must be

import MYmodule

Use getattr instead of .__dict__.get. Not only is it shorter, but it continue to work if you subclass Mymodule. That might not be your intention at this point, but using getattr is probably a good default habit anyway.

for funcs in dir(MYmodule.Mymodule):
   if funcs.startswith('MY'):
      result=getattr(MYmodule.Mymodule,funcs)(dataStructure)