Best way to display a Twitter feed (with history) on a Rails site

Question

On a Rails site, I'd like to display a certain Twitter feed, with pagination so the visitor can see previous tweets (as far back as needed).

I implemented it using the Twitter gem, using Search method, which has a nice pagination method, but hit a limitation that Twitter will only return the statutes from the last two weeks. So after going back a couple of pages, it won't fetch anymore.

I could use the user_timeline method, with max_id and then do my own pagination (passing the max_id of the last item viewed back to the controller to fetch the next batch).

Or, I could have a rake task that polls the Twitter feed frequently (with cron), and stores the tweets in the DB. The site would serve those up from the DB instead of querying Twitter at all.

Which would be the best or recommended method? I don't like having to store the Tweets in the DB, but that would also take care of the latency problem of querying Twitter (though I could use fragment caching to overcome that except that I haven't been able to get it working with Ajax).

Thanks for the advice.

Answer 1

I'd go for storing the tweets in the database.
So even if twitter is offline you won't have some long load problems. You'll just rely on your database and the tweets will be appropriately displayed.
Only your background job will fail because twitter is unavailable. But that's not really a problem.

Answer 2

I take the opposite view here, storing tweets in your database is not a good idea for a range of reasons.

1. you can never be sure that you got all the recently added tweets as a whole bunch could be added in quick succession. Sure, you can just make the cron job run more frequently, but then we get to the next problem.

2. if tweets are deleted, for whatever reason, your app will still cache them, which too me is not bad practice as they would have been removed for a reason.

To be honest, I would not have your app serve the tweets, but have a 'widget' (jquery or such) on the page which would love them once the page has loaded, and look at implementing some form of pagination there instead.

Answer 3

We download and store the tweets in a local database. I recently wrote a blog post about how we achieved this:

http://www.arctickiwi.com/blog/16-download-you-twitter-feed-using-ruby-on-rails-with-oauth

You can then use will_paginate to handle your pagination and go back as far as you want.