Dynamic Operator Overloading on dict classes in Python

Question

I have a class that dynamically overloads basic arithmetic operators like so...

import operator

class IshyNum:
    def __init__(self, n):
        self.num=n
        self.buildArith()

    def arithmetic(self, other, o):
        return o(self.num, other)

    def buildArith(self):
        map(lambda o: setattr(self, "__%s__"%o,lambda f: self.arithmetic(f, getattr(operator, o))), ["add", "sub", "mul", "div"])

if __name__=="__main__":
    number=IshyNum(5)
    print number+5
    print number/2
    print number*3
    print number-3

But if I change the class to inherit from the dictionary (class IshyNum(dict):) it doesn't work. I need to explicitly def __add__(self, other) or whatever in order for this to work. Why?

Answer 1

In general, never set __ methods on the instance -- they're only supported on the class. (In this instance, the problem is that they happen to work on old-style classes. Don't use old-style classes).

You probably want to use a metaclass, not the weird thing you're doing here.

Here's a metaclass tutorial: http://www.voidspace.org.uk/python/articles/metaclasses.shtml

Answer 2

I do not understand what you are trying to accomplish, but I am almost certain you are going about it in the wrong way. Some of my observations:

• I don't see why you're trying to dynamically generate those arithmetic methods. You don't do anything instance-specific with them, so I don't see why you would not just define them on the class.

  • The only reason they work at all is because IshyNum is an old-style class; this isn't a good thing, since old-style classes are long-deprecated and not as nice as new-style classes. (I'll explain later why you should be especially interested in this.)

  • If you wanted to automate the process of doing the same thing for multiple methods (probably not worth it in this case), you could just do this right after the class definition block.

    • Don't use map to do that. map is for making a list; using it for side effects is silly. Just use a normal for loop.

• If you want to use composition to refer lots of methods to the same attribute automatedly when using composition, use __getattr__ and redirect to that attribute's methods.

• Don't inherit dict. There is nothing much to gain from inheriting built-in types. It turns out it is more confusing than it's worth, and you don't get to re-use much.

  • If your code above is anything close to the stuff in your post, you really don't want to inherit dict. If it's not, try posting your real use case.

Here is what you really wanted to know:

• When you inherit dict, you are making a new-style class. IshyNum is an old-style class because it doesn't inherit object (or one of its subclasses).

  New-style classes have been Python's flagship kind of class for a decade and are what you want to use. In this case, they actually cause your technique no longer to work. This is fine, though, since there is no reason in the code you posted to set magic methods on a per-instance level and little reason ever to want to.

Answer 3

The answer is found in the two types of class that Python has.

The first code-snippet you provided uses a legacy "old-style" class (you can tell because it doesn't subclass anything - there's nothing before the colon). Its semantics are peculiar. In particular, you can add a special method to an instance:

class Foo:
   def __init__(self, num):
      self.num = num
      def _fn(other):
         return self.num + other.num
      self.__add__ = _fn

and get a valid response:

>>> f = Foo(2)
>>> g = Foo(1)
>>> f + g
3

But, subclassing dict means you are generating a new-style class. And the semantics of operator overloading are different:

class Foo (object):
   def __init__(self, num):
      self.num = num
      def _fn(other):
         return self.num + other.num
      self.__add__ = _fn
>>> f = Foo(2)
>>> g = Foo(1)
>>> f + g
Traceback ...
TypeError: unsupported operand type(s) for +: 'Foo' and 'Foo'

To make this work with new-style classes (which includes subclasses of dict or just about any other type you will find), you have to make sure the special method is defined on the class. You can do this through a metaclass:

class _MetaFoo(type):
    def __init__(cls, name, bases, args):
        def _fn(self, other):
            return self.num + other.num
        cls.__add__ = _fn

class Foo(object):
    __metaclass__ = _MetaFoo
    def __init__(self, num):
        self.num = num

>>> f = Foo(2)
>>> g = Foo(1)
>>> f+g
3

Also, the semantic difference means that in the very first case I could define my local add method with one argument (the self it uses is captured from the surrounding scope in which it is defined), but with new-style classes, Python expects to pass in both values explicitly, so the inner function has two arguments.

As a previous commenter mentioned, best to avoid old-style classes if possible and stick with new-style classes (old-style classes are removed in Python 3+). Its unfortunate that the old-style classes happened to work for you in this case, where new-style classes will require more code.

---

Edit:

You can also do this more in the way you originally tried by setting the method on the class rather than the instance:

class Foo(object):
    def __init__(self, num):
        self.num = num
setattr(Foo, '__add__', (lambda self, other: self.num + other.num))
>>> f = Foo(2)
>>> g = Foo(1)
>>> f+g
3

I'm afraid I sometimes think in Metaclasses, where simpler solutions would be better :)

Answer 4

For new-style classes, Python does not check the instance for an __add__ method when performing an addition, it checks the class instead. The problem is that you are binding the __add__ method (and all the others) to the instance as a bound method and not to the class as an unbound method. (This is true to other special methods as well, you can attach them only to the class, not to an instance). So, you'll probably want to use a metaclass to achieve this functionality (although I think this is a very awkward thing to do as it is much more readable to spell out these methods explicitly). Anyway, here is an example with metaclasses:

import operator

class OperatorMeta(type):
    def __new__(mcs, name, bases, attrs):
        for opname in ["add", "sub", "mul", "div"]:
            op = getattr(operator, opname)
            attrs["__%s__" % opname] = mcs._arithmetic_func_factory(op)
        return type.__new__(mcs, name, bases, attrs)

    @staticmethod
    def _arithmetic_func_factory(op):
        def func(self, other):
            return op(self.num, other)
        return func

class IshyNum(dict):
    __metaclass__ = OperatorMeta

    def __init__(self, n):
        dict.__init__(self)
        self.num=n

if __name__=="__main__":
    number=IshyNum(5)
    print number+5
    print number/2
    print number*3
    print number-3