Creating container relationship in declarative SQLAlchemy

Question

My Python / SQLAlchemy application manages a set of nodes, all derived from a base class Node. I'm using SQLAlchemy's polymorphism features to manage the nodes in a SQLite3 table. Here's the definition of the base Node class:

class Node(db.Base):
    __tablename__ = 'nodes'
    id = Column(Integer, primary_key=True)
    node_type = Column(String(40))
    title = Column(UnicodeText)
    __mapper_args__ = {'polymorphic_on': node_type}

and, as an example, one of the derived classes, NoteNode:

class NoteNode(Node):
    __mapper_args__ = {'polymorphic_identity': 'note'}
    __tablename__ = 'nodes_note'
    id = Column(None,ForeignKey('nodes.id'),primary_key=True)
    content_type = Column(String)
    content = Column(UnicodeText)

Now I need a new kind of node, ListNode, that is an ordered container of zero or more Nodes. When I load a ListNode, I want it to have its ID and title (from the base Node class) along with a collection of its contained (child) nodes. A Node may appear in more than one ListNode, so it's not a proper hierarchy. I would create them along these lines:

note1 = NoteNode(title=u"Note 1", content_type="text/text", content=u"I am note #1")
session.add(note1)

note2 = NoteNode(title=u"Note 2", content_type="text/text", content=u"I am note #2")
session.add(note2)

list1 = ListNode(title=u"My List")
list1.items = [note1,note2]
session.add(list1)

The list of children should only consist of Node objects -- that is, all I need is their base class stuff. They shouldn't be fully realized into the specialized classes (so I don't get the whole graph at once, among other reasons).

I started along the following lines, cobbling together bits and pieces I found in various places without a complete understanding of what was going on, so this may not make much sense:

class ListNode(Node):
    __mapper_args__ = {'polymorphic_identity': 'list', 'inherit_condition':id==Node.id}
    __tablename__ = 'nodes_list_contents'
    id = Column(None, ForeignKey('nodes.id'), primary_key=True)
    item_id = Column(None, ForeignKey('nodes.id'), primary_key=True)
    items = relation(Node, primaryjoin="Node.id==ListNode.item_id")

This approach fails in several ways: it doesn't appear to allow an empty ListNode, and setting the items attribute to a list results in SQLAlchemy complaining that 'list' object has no attribute '_sa_instance_state'. Not surprisingly, hours of random mutations on this theme haven't given any good results,

I have limited experience in SQLAlchemy but really want to get this working soon. I'd very much appreciate any advice or direction you can offer. Thanks in advance!

Answer 1

You need an additional table for many-to-many relation:

nodes_list_nodes = Table(
    'nodes_list_nodes', metadata,
    Column('parent_id', None, ForeignKey('nodes_list.id'), nullable=False),
    Column('child_id', None, ForeignKey(Node.id), nullable=False),
    PrimaryKeyConstraint('parent_id', 'child_id'),
)

class ListNode(Node):
    __mapper_args__ = {'polymorphic_identity': 'list'}
    __tablename__ = 'nodes_list'
    id = Column(None, ForeignKey('nodes.id'), primary_key=True)
    items = relation(Node, secondary=nodes_list_nodes)

Update: below is an example for ordered list using association_proxy:

from sqlalchemy.orm.collections import InstrumentedList
from sqlalchemy.ext.associationproxy import association_proxy


class ListNodeAssociation(Base):
    __tablename__ = 'nodes_list_nodes'
    parent_id = Column(None, ForeignKey('nodes_list.id'), primary_key=True)
    child_id = Column(None, ForeignKey(Node.id), primary_key=True)
    order = Column(Integer, nullable=False, default=0)
    child = relation(Node)
    __table_args__ = (
        PrimaryKeyConstraint('parent_id', 'child_id'),
        {},
    )


class OrderedList(InstrumentedList):

    def append(self, item):
        if self:
            item.order = self[-1].order+1
        else:
            item.order = 1
        InstrumentedList.append(self, item)


class ListNode(Node):
    __mapper_args__ = {'polymorphic_identity': 'list'}
    __tablename__ = 'nodes_list'
    id = Column(None, ForeignKey('nodes.id'), primary_key=True)
    _items = relation(ListNodeAssociation,
                      order_by=ListNodeAssociation.order,
                      collection_class=OrderedList,
                      cascade='all, delete-orphan')
    items = association_proxy(
                '_items', 'child',
                creator=lambda item: ListNodeAssociation(child=item))