Why does python use 'magic methods'?

Question

I've been playing around with Python recently, and one thing I'm finding a bit odd is the extensive use of 'magic methods', e.g. to make its length available an object implements a method def __len__(self) and then it is called when you write len(obj).

I was just wondering why objects don't simply define a len(self) method and have it called directly as a member of the object, e.g. obj.len()? I'm sure there must be good reasons for Python doing it the way it does, but as a newbie I haven't worked out what they are yet.

Answer 1

AFAIK, len is special in this respect and has historical roots.

Here's a quote from the FAQ:

Why does Python use methods for some functionality (e.g. list.index()) but functions for other (e.g. len(list))?

The major reason is history. Functions were used for those operations that were generic for a group of types and which were intended to work even for objects that didn’t have methods at all (e.g. tuples). It is also convenient to have a function that can readily be applied to an amorphous collection of objects when you use the functional features of Python (map(), apply() et al).

In fact, implementing len(), max(), min() as a built-in function is actually less code than implementing them as methods for each type. One can quibble about individual cases but it’s a part of Python, and it’s too late to make such fundamental changes now. The functions have to remain to avoid massive code breakage.

The other "magical methods" (actually called special method in the Python folklore) make lots of sense, and similar functionality exists in other languages. They're mostly used for code that gets called implicitly when special syntax is used.

For example:

• overloaded operators (exist in C++ and others)
• constructor/destructor
• hooks for accessing attributes
• tools for metaprogramming

and so on...

Answer 2

From the Zen of Python:

In the face of ambiguity, refuse the temptation to guess.
There should be one-- and preferably only one --obvious way to do it.

This is one of the reasons - with custom methods, developers would be free to choose a different method name, like getLength(), length(), getlength() or whatsoever. Python enforces strict naming so that the common function len() can be used.

All operations that are common for many types of objects are put into magic methods, like __nonzero__, __len__ or __repr__. They are mostly optional, though.

Operator overloading is also done with magic methods (e.g. __le__), so it makes sense to use them for other common operations, too.

Answer 3

They are not really "magic names". It's just the interface an object has to implement to provide a given service. In this sense, they are not more magic than any predefined interface definition you have to reimplement.

Answer 4

Some of these functions do more than a single method would be able to implement (without abstract methods on a superclass). For instance bool() acts kind of like this:

def bool(obj):
    if hasattr(obj, '__bool__'):
        return bool(obj.__nonzero__())
    elif hasattr(obj, '__len__'):
        if obj.__len__():
            return True
        else:
            return False
    return True

You can also be 100% sure that bool() will always return True or False; if you relied on a method you couldn't be entirely sure what you'd get back.

Some other functions that have relatively complicated implementations (more complicated than the underlying magic methods are likely to be) are iter() and cmp(), and all the attribute methods (getattr, setattr and delattr). Things like int also access magic methods when doing coercion (you can implement __int__), but do double duty as types. len(obj) is actually the one case where I don't believe it's ever different from obj.__len__().

Answer 5

There is not a lot to add to the above two posts, but all the "magic" functions are not really magic at all. They are part of the __ builtins__ module which is implicitly/automatically imported when the interpreter starts. IE:

from __builtins__ import *

happens every time before your program starts.

I always thought it would be more correct if python only did this for the interactive shell, and required scripts to import the various parts from builtins they needed. Also probably different __ main__ handling would be nice in shells vs interactive. Anyways, check out all the functions, and see what it is like without them:

dir (__builtins__)
...
del __builtins__