Get tweets of a public twitter profile

Question

Hi folks,

I have a list of usernames on Twitter whose profiles are public. I wish to get "all the tweets" they have posted from the day they formed their profile. I checked Twitter4J examples http://github.com/yusuke/twitter4j/blob/master/twitter4j-examples/src/main/java/twitter4j/examples/GetTimelines.java . According to the Twitter API documentation, only the 20 most recent tweets are returned. Is there anyway I could perform my task?

Answer 1

If you read through http://dev.twitter.com/doc/get/statuses/user_timeline, you can retrieve up to 200 tweets at a time if you specify "count=200" in your API request.

You can also use "page=x" to get different paginated results; you could keep doing this until you've retrieved every tweet the user has posted.

I'm not sure how your Java application would create this, but your requests would probably look like:

http://api.twitter.com/1/statuses/user_timeline.xml?id=SomeUsername&count=200&page=1
http://api.twitter.com/1/statuses/user_timeline.xml?id=SomeUsername&count=200&page=2
http://api.twitter.com/1/statuses/user_timeline.xml?id=SomeUsername&count=200&page=3

... etc.

Keep in mind that these requests are rate-limited so you'd need to be careful not to exceed the limit.

Answer 2

To use Twitter4J to get all posts from a user you'll have to make your request over multiple pages..

Below code based off an example from http://github.com/yusuke/twitter4j/blob/master/twitter4j-examples/src/main/java/twitter4j/examples/GetTimelines.java

Twitter unauthenticatedTwitter = new TwitterFactory().getInstance();
//First param of Paging() is the page number, second is the number per page (this is capped around 200 I think.
Paging paging = new Paging(1, 100);
List<Status> statuses = unauthenticatedTwitter.getUserTimeline("google",paging);

Just loop and keep grabbing new pages until there are no new posts should work.