How would you design a twitter like message system, how to design the messaging system?

Question

If you were to create an application like twitter, how would you go about designing the messaging system?

Specifically look for ideas on the basic data model, and how you would write the method that takes the user's tweet and then sends it out to all its followers?

example:

Tweets ( tweetID, userID, message, datesend)
User (userID, ...)
Followers(userID, followerUserID)

Inbox(userID, tweetID)

is the above model a good starting point?

Would you first insert the tweet, then push a message to the queue. Then one-by-one, take a message off the queue and push the message to its subscribers?

(I am ignoring the mobile functionality of twitter, just focussing on the web based functionality, but I thought of using a queue from the start so one could add other functionality later)

Answer 1

I don't think twitter uses a message queue (inbox in your data model.) I think in twitter everything is done via date. Thus each user has a "last viewed date" and the inbox/message queue is created by looking for all subscribed tweets after that last viewed date.

So to modify your data model, remove inbox and add in a last view date column to user.

Also I would expect that follower information is not stored as who follows a user but instead as which users a give user is following. Of course it could be stored both ways but that seems to make more sense to me.

Answer 2

The High Scalability blog has a number of articles on twitter and it's infrastructure and changes over time that might interest you.