Is there an API to determine the most common link in a number of tweets?

Question

Is there an API (Twitter API does not provide this) that I can use to determine the most common links in 200 tweets for example. What I want to do is to get the latest 200 tweets and then determine what are people talking about, I am sure that the tweets will contain links (because I will ask the twitter API to return tweets that contain links only) but I will also want to make sure that my code will understand that Two URLs are the same even if they have different bit.ly links.

What I am trying to do (this might make it easier for your guys to provide some help) is that i am trying to determine what is the most important subject people are talking about in these 200 tweets. I understand that people might be talking about the same story but provide different links, however, i am not sure if there is an easy way to understand that.

Links to examples, APIs, sample code, and any other ideas will be helpful :)

If you need more information to explain this please tell me and I will edit the question to include more information

Answer 1

Not that I know of, but you can accomplish this by..

1. Find all of the links in the list of tweets using a regex pattern.

2. Use the twitter search api to search for each link. The number of results is returned.

3. Manually sort the links by # of results returned.

Answer 2

Fundamentally you can get this from the api, first get the latest public timeline (this will be 100 tweets, if you need 200 then you need to request a cursor and create a loop that checks if the next_cursor value is greater than 0) and then build a spider that determines relevancy.

http://api.twitter.com/1/statuses/public_timeline.???

where ??? is json, xml, rss or atom

If you want to determine the popularity of words then dump all the text into a string and then split it on spaces, punctuation etc, discard non-nouns, sort it and create a dictionary variable with the words and the count of the words.

If you want to determine the popularity of links then it is the same process but with an extra step to do a web request on each link to determine the ultimate link destination.

Answer 3

Building on what others say, you can use twitter search to get the tweets no problem and I wont go into that part in this answer.

A possible route for the short links:

You could, for example, goto bit.ly and create a custom short link for the url you are wanting to keep track of. Using that link if you add a + to the end of the url you will get link stats. example: http://bit.ly/tweelay+ Additionally, bit.ly keeps track of other short links that point to the same url. Which you could then use in your searches.

using bit.ly /stats API you can get a list of the shorten urls.

depending on the urls you are trying to keep track of you may have access to referral logs. (i.e. your own website) Using your referral log you may also be able to find additional short urls that you can use to search.