Python's use of __new__ and __init__ ?

Question

I'm just trying to streamline one of my classes and have introduced some functionality in the same style as the flyweight design pattern.

However, I'm a bit confused as to why __init__ is always called after __new__. I wasn't expecting this. Can anyone tell me why this is happening and how I implement this functionality otherwise? (apart from putting the implementation into the __new__ which feels quite hacky).

Here's an example:

class A(object):
    _dict = dict()

    def __new__(cls):
     if 'key' in A._dict:
      print "EXISTS"
      return A._dict['key']
     else:
      print "NEW"
      return super(A, cls).__new__(cls)

    def __init__(self):
     print "INIT"
     A._dict['key'] = self
     print ""

a1 = A()
a2 = A()
a3 = A()

Outputs:

NEW
INIT

EXISTS
INIT

EXISTS
INIT

Why?

Answer 1

__new__ is static class method, while __init__ is instance method. __new__ has to create the instance first, so __init__ can initialize it. Note that __init__ takes self as parameter. Until you create instance there is no self.

Now, I gather, that you're trying to implement singleton pattern in Python. There are a few ways to do that.

Also, as of Python 2.6, you can use class decorators.

def singleton(cls):
    instances = {}
    def getinstance():
        if cls not in instances:
            instances[cls] = cls()
        return instances[cls]
    return getinstance

@singleton
class MyClass:
  ...

Answer 2

Use __new__ when you need to control the creation of a new instance. Use __init__ when you need to control initialization of a new instance.

__new__ is the first step of instance creation. It's called first, and is responsible for returning a new instance of your class. In contrast, __init__ doesn't return anything; it's only responsible for initializing the instance after it's been created.

In general, you shouldn't need to override __new__ unless you're subclassing an immutable type like str, int, unicode or tuple.

From: http://mail.python.org/pipermail/tutor/2008-April/061426.html

You should consider that what you are trying to do is usually done with a Factory and that's the best way to do it. Using __new__ is not a good clean solution so please consider the usage of a factory. Here you have a good factory example.

Answer 3

However, I'm a bit confused as to why __init__ is always called after __new__.

Not much of a reason other than that it just is done that way. __new__ doesn't have the responsibility of initializing the class, some other method does (__call__, possibly-- I don't know for sure).

I wasn't expecting this. Can anyone tell me why this is happening and how I implement this functionality otherwise? (apart from putting the implementation into the __new__ which feels quite hacky).

You could have __init__ do nothing if it's already been initialized, or you could write a new metaclass with a new __call__ that only calls __init__ on new instances, and otherwise just returns __new__(...).

Answer 4

__new__ should return a new, blank instance of a class. __init__ is then called to initialise that instance. You're not calling __init__ in the "NEW" case of __new__, so it's being called for you. The code that is calling __new__ doesn't keep track of whether __init__ has been called on a particular instance or not nor should it, because you're doing something very unusual here.

You could add an attribute to the object in the __init__ function to indicate that it's been initialised. Check for the existence of that attribute as the first thing in __init__ and don't proceed any further if it has been.

Answer 5

To quote the documentation:

"Typical implementations create a new instance of the class by invoking the superclass's __new__() method using "super(currentclass, cls).__new__(cls[, ...])"with appropriate arguments and then modifying the newly-created instance as necessary before returning it."

...

"If __new__() does not return an instance of cls, then the new instance's __init__() method will not be invoked.

__new__() is intended mainly to allow subclasses of immutable types (like int, str, or tuple) to customize instance creation."