How do I join three tables with SQLalchemy and keeping all of the columns in one of the tables?

Question

So, I have three tables:

The class defenitions:

engine = create_engine('sqlite://test.db', echo=False)
SQLSession = sessionmaker(bind=engine)
Base = declarative_base()

class Channel(Base):
    __tablename__ = 'channel'

    id = Column(Integer, primary_key = True)
    title = Column(String)
    description = Column(String)
    link = Column(String)
    pubDate = Column(DateTime)

class User(Base):
    __tablename__ = 'user'

    id = Column(Integer, primary_key = True)
    username = Column(String)
    password = Column(String)
    sessionId = Column(String)

class Subscription(Base):
    __tablename__ = 'subscription'

    userId = Column(Integer, ForeignKey('user.id'), primary_key=True)
    channelId = Column(Integer, ForeignKey('channel.id'), primary_key=True)

NOTE: I know user.username should be unique, need to fix that, and I'm not sure why SQLalchemy creates some row names with the double-quotes.

And I'm trying to come up with a way to retrieve all of the channels, as well as an indication on what channels one particular user (identified by user.sessionId together with user.id) has a subscription on.

For example, say we have four channels: channel1, channel2, channel3, channel4; a user: user1; who has a subscription on channel1 and channel4. The query for user1 would return something like:

channel.id | channel.title | subscribed
---------------------------------------
1            channel1        True
2            channel2        False
3            channel3        False
4            channel4        True

This is a best-case result, but since I have absolutely no clue as how to accomplish the subscribed column, I've been instead trying to get the particular users id in the rows where the user has a subscription and where a subscription is missing, just leave it blank.

The database engine that I'm using together with SQLalchemy atm. is sqlite3

I've been scratching my head over this for two days now, I've no problem joining together all three by way of the subscription table but then all of the channels where the user does not have a subscription gets omitted.

I hope I've managed to describe my problem sufficiently, thanks in advance.

EDIT: Managed to solve this in a slightly clunky way involving a sub-query:

# What a messy SQL query!
stmt = query(Subscription).filter_by(userId = uid()).join((User, Subscription.userId == User.id)).filter_by(sessionId = id()).subquery()
subs = aliased(Subscription, stmt)
results = query(Channel.id, Channel.title, subs.userId).outerjoin((subs, subs.channelId == Channel.id))

However, I'll be continuing to search for a more elegant solution, so answers are still very much welcomed.

Answer 1

To make this a little easyer I've added relationships to your model, that way you can just do user.subscriptions to get all the subscriptions.

engine = create_engine('sqlite://test.db', echo=False)
SQLSession = sessionmaker(bind=engine)
Base = declarative_base()

class Channel(Base):
    __tablename__ = 'channel'

    id = Column(Integer, primary_key = True)
    title = Column(String)
    description = Column(String)
    link = Column(String)
    pubDate = Column(DateTime)

class User(Base):
    __tablename__ = 'user'

    id = Column(Integer, primary_key = True)
    username = Column(String)
    password = Column(String)
    sessionId = Column(String)

class Subscription(Base):
    __tablename__ = 'subscription'

    userId = Column(Integer, ForeignKey('user.id'), primary_key=True)
    user = relationship(User, primaryjoin=userId == User.id, backref='subscriptions')
    channelId = Column(Integer, ForeignKey('channel.id'), primary_key=True)
    channel = relationship(channel, primaryjoin=channelId == channel.id, backref='subscriptions')

results = session.query(
    Channel.id,
    Channel.title,
    Channel.subscriptions.any().label('subscribed'),
)

for channel in results:
    print channel.id, channel.title, channel.subscribed

Answer 2

Don't query from the user. Query from the Channel.

user = query(User).filter_by(id=1).one()
for channel in query(Channel).all():
    print channel.id, channel.title, user in channel.subscriptions.user

That way you get all the channels, not just the ones that are associated with the user in question.

Answer 3

Option-1:

Subscription is just a many-to-many relation object, and I would suggest that you model it as such rather then as a separate class. See Configuring Many-to-Many Relationships documentation of SQLAlchemy/declarative.

You model with the test code becomes:

from sqlalchemy import create_engine, Column, Integer, DateTime, String, ForeignKey, Table
from sqlalchemy.orm import relation, scoped_session, sessionmaker, eagerload
from sqlalchemy.ext.declarative import declarative_base

engine = create_engine('sqlite:///:memory:', echo=True)
session = scoped_session(sessionmaker(bind=engine, autoflush=True))
Base = declarative_base()

t_subscription = Table('subscription', Base.metadata,
    Column('userId', Integer, ForeignKey('user.id')),
    Column('channelId', Integer, ForeignKey('channel.id')),
)

class Channel(Base):
    __tablename__ = 'channel'

    id = Column(Integer, primary_key = True)
    title = Column(String)
    description = Column(String)
    link = Column(String)
    pubDate = Column(DateTime)

class User(Base):
    __tablename__ = 'user'

    id = Column(Integer, primary_key = True)
    username = Column(String)
    password = Column(String)
    sessionId = Column(String)

    channels = relation("Channel", secondary=t_subscription)

# NOTE: no need for this class
# class Subscription(Base):
    # ...

Base.metadata.create_all(engine)


# ######################
# Add test data
c1 = Channel()
c1.title = 'channel-1'
c2 = Channel()
c2.title = 'channel-2'
c3 = Channel()
c3.title = 'channel-3'
c4 = Channel()
c4.title = 'channel-4'
session.add(c1)
session.add(c2)
session.add(c3)
session.add(c4)
u1 = User()
u1.username ='user1'
session.add(u1)
u1.channels.append(c1)
u1.channels.append(c3)
u2 = User()
u2.username ='user2'
session.add(u2)
u2.channels.append(c2)
session.commit()


# ######################
# clean the session and test the code
session.expunge_all()

# retrieve all (I assume those are not that many)
channels = session.query(Channel).all()

# get subscription info for the user
#q = session.query(User)
# use eagerload(...) so that all 'subscription' table data is loaded with the user itself, and not as a separate query
q = session.query(User).options(eagerload(User.channels))
for u in q.all():
    for c in channels:
        print (c.id, c.title, (c in u.channels))

which produces following output:

(1, u'channel-1', True)
(2, u'channel-2', False)
(3, u'channel-3', True)
(4, u'channel-4', False)
(1, u'channel-1', False)
(2, u'channel-2', True)
(3, u'channel-3', False)
(4, u'channel-4', False)

Please note the use of eagerload, which will issue only 1 SELECT statement instead of 1 for each User when channels are asked for.

Option-2:

But if you want to keep you model and just create an SA query that would give you the columns as you ask, following query should do the job:

from sqlalchemy import and_
from sqlalchemy.sql.expression import case
#...
q = (session.query(#User.username, 
                   Channel.id, Channel.title, 
                   case([(Subscription.channelId == None, False)], else_=True)
                  ).outerjoin((Subscription, 
                                and_(Subscription.userId==User.id, 
                                     Subscription.channelId==Channel.id))
                             )
    )
# optionally filter by user
q = q.filter(User.id == uid()) # assuming uid() is the function that provides user.id
q = q.filter(User.sessionId == id()) # assuming uid() is the function that provides user.sessionId
res = q.all()
for r in res:
    print r

The output is absolutely the same as in the option-1 above.