Python std methods hierarchy calls documented ?

Question

Hi,

just encountered a problem at dict "type" subclassing. I did override __iter__ method and expected it will affect other methods like iterkeys, keys etc. because I believed they call __iter__ method to get values but it seems they are implemented independently and I have to override all of them.

Is this a bug or intention they don't make use of other methods and retrieves values separately ?

I didn't find in the standard Python documentation description of calls dependency between methods of standard classes. It would be handy for sublassing work and for orientation what methods is required to override for proper behaviour. Is there some supplemental documentation about python base types/classes internals ?

Answer 1

If not specified in the documentation, it is implementation specific. Implementations other that CPython might re-use the iter method to implement iterkeys and others. I would not consider this to be a bug, but simply a bit of freedom for the implementors.

I suspect there is a performance factor in implementing the methods independently, especially as dictionaries are so widely used in Python.

So basically, you should implement them.

Answer 2

Subclass Mapping or MuteableMapping from the collections module instead of dict and you get all those methods for free.

Here is a example of a minimal mapping and some of the methods you get for free:

import collections
class MinimalMapping(collections.Mapping):
    def __init__(self, *items ):
        self.elements = dict(items)
    def __getitem__(self, key):
        return self.elements[key]
    def __len__(self):
        return len(self.elements)
    def __iter__(self):
        return iter(self.elements)

t = MinimalMapping()
print (t.iteritems, t.keys, t.itervalues, t.get)

To subclass any of the builtin containers you should always use the appropriate baseclass from the collections module.

Answer 3

You know the saying: "You know what happens when you assume." :-)

They don't officially document that stuff because they may decide to change it in the future. Any unofficial documentation you may find would simply document the current behavior of one Python implementation, and relying on it would result in your code being very, very fragile.

When there is official documentation of special methods, it tends to describe behavior of the interpreter with respect to your own classes, such as using __len__() when __nonzero__() isn't implemented, or only needing __lt()__ for sorting.

Since Python uses duck typing, you usually don't actually need to inherit from a built-in class to make your own class act like one. So you might reconsider whether subclassing dict is really what you want to do. You might choose a different class, such as something from the collections module, or to encapsulate rather than inheriting. (The UserString class uses encapsulation, since you can't subclass strings.) Or just start from scratch.

Answer 4

Instead of subclassing dict, you could instead just create make your own class that has exactly the properties you want without too much trouble. Here's a blog post with an example of how to do this. The __str__() method in it isn't the greatest, but that's easily corrected the rest provide the functionality you seek.