Pythonic solution to my reduce getattr problem

Question

I used to use reduce and getattr functions for calling attributes in a chain way like "thisattr.thatattr.blaattar" IE:

reduce(getattr, 'xattr.yattr.zattr'.split('.'), myobject)

Works perfectly fine, however now I have a new requirement, my strings can call for a specific number of an attribute as such: "thisattr.thatattr[2].blaattar"

reduce(getattr, 'xattr.yattr[2].zattr'.split('.'), myobject)

Now it doesn't work, I get xattr object has no attribute 'yattr[2]' error.

What would be an elegent solution to this, which works for either way ?

Regards

Answer 1

You will need to

1. Get xattr.yattr
2. Get the second item of that
3. Of the second item, get zattr

As you can see, this involved two different operations. reduce can't do that (elegantly). A solution working for both would have to parse the string to detect where indexed access is needed. A simple but fragile (i.e. behaves undefined if fed BS) solution would look like:

def extended_chain_getattr(names, obj):
    import re
    result = obj        
    for name in names.split('.'):
        name_match = re.match(r'([a-zA-Z_][a-zA-Z0-9_]*)(\[\d\])?', name)
        assert name_match is not None
        result = getattr(result, name_match.group(1))
        if len(name_match.groups()) == 2:
            index = int(name_match.group(2))
            result = result[index]
    return result

Off the top of my head, therefore untested.

Answer 2

And later you could wish to call some method rather than getting attribute. Re-implementing parts of python approach quickly will become a nightmare. Even current requirement of getattr/getitem support cannot be solved as one-liner.

Instead, you could just use python itself to interpret python,

# Create some object for testing
>>> class A(object):
...     b = None
... 
>>> a = A()
>>> a.b = A()
>>> a.b.b = A()
>>> a.b.b.b = [A(), A(), A(), A()]
>>> a.b.b.b[1].b
>>> a.b.b.b[1].b = "Some result"
>>> 
>>> ctx = {'obj':a, 'val':None}
>>> exec("val = obj.{0}".format('b.b.b[1].b')) in ctx
>>> ctx['val']
'Some result'

Answer 3

What you are asking for seems quite difficult as you want to mix attribute selection with method calls (as the index is just sugar for a call). Calling functions is easy enough to do by using getattr to give you a bound method, but then you need to convert the part of the string that contains the arguments into the actual arguments.

Given that you will need an eval() to compute the arguments anyway, why not just eval the whole thing?

def proc(objname, attrstring ) :
  return eval( '%s.%s' % (objname,attrstring) )

Your example is then:

proc("myobject", "xattr.yattr[2].zattr")

Answer 4

You could try:

import re
extended_split = re.compile(r'''\[\d+\]|[^\[.]+''').findall

def extended_getattr(obj, comp):
    if comp[0] == '[':
        return obj[int(comp[1:-1])]
    else:
        return getattr(obj, comp)

reduce(extended_getattr, extended_split('xattr.yattr[2].zattr'), myobject)

Note that it assumes the stuff inside […] is a nonnegative decimal number.

---

In case you concern about performance, it is still faster than eval in my test:

~:491$ python -m timeit -s 'from z import f1, f3, f, rs' 'f3(rs, "f")'   # eval
100 loops, best of 3: 5.62 msec per loop

~:492$ python -m timeit -s 'from z import f1, f3, f, rs' 'f1(rs, f)'     # my method
100 loops, best of 3: 4.69 msec per loop

Content of z.py:

import re
import random
from functools import reduce

extended_split = re.compile(r'''\[\d+\]|[^\[.]+''').findall

def extended_getattr(obj, comp):
    if comp[0] == '[':
        return obj[int(comp[1:-1])]
    else:
        return getattr(obj, comp)

class Foo(object):
    def __init__(self):
        self.foo = self

    def __getitem__(self, i):
        return self

def construct_random_string():
    yield 'foo'
    for i in range(2000):
        if random.randrange(2):
            yield '.foo'
        else:
            yield '[0]'


random.seed(0)  # to ensure fair comparison
rs = ''.join(construct_random_string())

f = Foo()

def f1(names, obj):
    return reduce(extended_getattr, extended_split(names), obj)

def f3(attrstring, objname) :
    return eval( '%s.%s' % (objname, attrstring) )

Answer 5

Here is a tiny parser to handle slice and nested list notation:

# define class that we can just add attributes to
class Bag(object): pass

z = Bag()
z.xattr = Bag()
z.xattr.yattr = [Bag(), Bag(), Bag()]
z.xattr.yattr[2].zattr = 100
z.xattr.yattr[1] = [0,1,2,3,4,5]

from pyparsing import *

LBRACK,RBRACK = map(Suppress,'[]')
ident = Word(alphas+"_", alphanums+"_")
integer = Word(nums+'-',nums).setParseAction(lambda t:int(t[0]))
NONE = Literal("None").setParseAction(replaceWith(None))
indexref = LBRACK + Group(delimitedList((Optional(integer|NONE,None)), delim=':')) + RBRACK
compoundAttr = delimitedList(Group(ident("name") + ZeroOrMore(indexref)("index")), delim='.')

def lookup(ob, attr):
    try:
        attrParts = compoundAttr.parseString(attr)
    except ParseException:
        raise AttributeError("could not resolve compound attribute '%s'" % attr)

    # remaining code will raise AttributeError or IndexError as appropriate

    ret = ob
    for a in attrParts:
        ret = getattr(ret, a.name)
        if a.index:
            for i in a.index:
                if len(i) == 1:
                    ret = ret[i[0]]
                else:
                    ret = ret[slice(*i.asList())]
    return ret


print len(lookup(z, 'xattr.yattr'))
print len(lookup(z, 'xattr.yattr[1:3]'))
print len(lookup(z, 'xattr.yattr[None:3]'))
print lookup(z, 'xattr.yattr[1][None:4]')
print sum(lookup(z, 'xattr.yattr[1][:4]'))
print lookup(z, 'xattr.yattr[2].zattr')