Twitter friends timeline not returning full history

Question

I am using twitter4J to get a user's friends timeline, but it is not returning the full available history. I know there are pagination limits (200 per request and 3200 total as per http://bit.ly/ck8ysq) and I am well within those. I make a request like so:

private static final int MAX_COUNT = 200;

private List<Status> getAllStatuses(long sinceID) throws TwitterException {
    Twitter twitter = new Twitter(username, password);
    List<Status> friendsTimelineList = new ArrayList<Status>();
    List<Status> tempList;

    int page = 0;
    do {
        page++;
        tempList = twitter.getFriendsTimeline(
                             new Paging(page, MAX_COUNT, sinceID));
        if(tempList == null )
            break;
        friendsTimelineList.addAll(tempList);
    } while(true);

    return friendsTimelineList;
}

This results in only 423 statuses being returned across 3 pages. Any idea why this might be happening?

Answer 1

Basically there is no call that I know of in the API that will allow you to get historical data from Twitter, past a certain amount of data. Some speculate that they do not even record the information in their own servers. Digging through the different java api's at least that are available, there was nothing along the lines of returning more then a certain amount of pages.

There is another question (similar) posted here where I got that information from: http://stackoverflow.com/questions/1662151/getting-historical-data-from-twitter

So to answer your question it is happening that way because Twitter doesn't allow you to grab more then the, either because they don't have it, or they don't want you to have it:)