Subclassed django models with integrated querysets

Question

Like in this question, except I want to be able to have querysets that return a mixed body of objects:

>>> Product.objects.all()
[<SimpleProduct: ...>, <OtherProduct: ...>, <BlueProduct: ...>, ...]

I figured out that I can't just set Product.Meta.abstract to true or otherwise just OR together querysets of differing objects. Fine, but these are all subclasses of a common class, so if I leave their superclass as non-abstract I should be happy, so long as I can get its manager to return objects of the proper class. The query code in django does its thing, and just makes calls to Product(). Sounds easy enough, except it blows up when I override Product.__new__, I'm guessing because of the __metaclass__ in Model... Here's non-django code that behaves pretty much how I want it:

class Top(object):
    _counter = 0
    def __init__(self, arg):
        Top._counter += 1
        print "Top#__init__(%s) called %d times" % (arg, Top._counter)
class A(Top):
    def __new__(cls, *args, **kwargs):
        if cls is A and len(args) > 0:
            if args[0] is B.fav:
                return B(*args, **kwargs)
            elif args[0] is C.fav:
                return C(*args, **kwargs)
            else:
                print "PRETENDING TO BE ABSTRACT"
                return None # or raise?
        else:
            return super(A).__new__(cls, *args, **kwargs)
class B(A):
    fav = 1
class C(A):
    fav = 2
A(0) # => None
A(1) # => <B object>
A(2) # => <C object>

But that fails if I inherit from django.db.models.Model instead of object:

File "/home/martin/beehive/apps/hello_world/models.py", line 50, in <module>
    A(0)
TypeError: unbound method __new__() must be called with A instance as first argument (got ModelBase instance instead)

Which is a notably crappy backtrace; I can't step into the frame of my __new__ code in the debugger, either. I have variously tried super(A, cls), Top, super(A, A), and all of the above in combination with passing cls in as the first argument to __new__, all to no avail. Why is this kicking me so hard? Do I have to figure out django's metaclasses to be able to fix this or is there a better way to accomplish my ends?

Answer 1

Okay, this works: https://gist.github.com/348872

The tricky bit was this.

class A(Top):
    pass

def newA(cls, *args, **kwargs):
    # [all that code you wrote for A.__new__]

A.__new__ = staticmethod(newA)

Now, there's something about how Python binds __new__ that I maybe don't quite understand, but the gist of it is this: django's ModelBase metaclass creates a new class object, rather than using the one that's passed in to its __new__; call that A_prime. Then it sticks all the attributes you had in the class definition for A on to A_prime, but __new__ doesn't get re-bound correctly.

Then when you evaluate A(1), A is actually A_prime here, python calls <A.__new__>(A_prime, 1), which doesn't match up, and it explodes.

So the solution is to define your __new__ after A_prime has been defined.

Maybe this is a bug in django.db.models.base.ModelBase.add_to_class, maybe it's a bug in Python, I don't know.

Now, when I said "this works" earlier, I meant this works in isolation with the minimal object construction test case in the current SVN version of Django. I don't know if it actually works as a Model or is useful in a QuerySet. If you actually use this in production code, I will make a public lightning talk out of it for pdxpython and have them mock you until you buy us all gluten-free pizza.

Answer 2

Basically what you're trying to do is to return the different child classes, while querying a shared base class. That is: you want the leaf classes. Check this snippet for a solution: http://www.djangosnippets.org/snippets/1034/

Also be sure to check out the docs on Django's Contenttypes framework: http://docs.djangoproject.com/en/dev/ref/contrib/contenttypes/ It can be a bit confusing at first, but Contenttypes will solve additional problems you'll probably face when using non-abstract base classes with Django's ORM.