Facebook crawler?

Question

Hi all,

I'm a graduate student whose research is complex network. I am working on a project that involves analyzing connections between Facebook users. Is it possible to write a crawler for Facebook based on friendship information?

I looked around but couldn't find any things useful so far. It seems Facebook isn't fond of such activity. Can I rely on the Facebook API?

Thanks

Update (Jan-08-2010): Thank you very much for the responses. I guess I probably need to contact Facebook directly then. Cheers

Answer 1

The problem is that facebook friendship information is typically private and only accessible to friends. It should be a lot easier to build this network on Twitter, if this is an option for you.

Answer 2

You can't rely on the Facebook API unfortunately. To get friend information, you need to use something like friends.get(). However, any Facebook API method that returns user information like this requires that you have an active session key from that user, and generally the way you get an active session key is to have the user come to your Facebook application or page.

In summary, the information you are talking about is essentially private. You can't pick a person from Facebook, get their friends, and get those friend's friends, and so on. To me this is a good thing for privacy, but of course it prevents arbitrary analysis.

I'd throw out the idea of writing a quick and dirty application with some user appeal that you could use for research. If a group like S**t My Dad Says (funny, not really safe for work) can get 120,000 users in a couple of months, you could probably plead your case with a small research application and get a reasonable amount of users.

Answer 3

As others have stated, this is typically private information. If, however, Facebook per se isn't a requirement, you could use Google's Social API. A snippet from the Google Social Graph API page: "With the Social Graph API, developers can now utilize public connections their users have already created in other web services. It makes information about public connections between people easily available and useful."

Here's an article on using it in Ruby: http://articles.sitepoint.com/article/google-social-graph-api-ruby-rails#

Answer 4

Hi,

I have a very similar need (I am a graduate student researching in social networking/semantic web). I was wondering, have you been able to get the required data from Facebook or from any other possible source?

Thank you..

Answer 5

Hi, I am a student at the University of Venice and i'm writing a facebook crawler but i've a bad news for you...the other posts are true...you must write a classic crawler without the help of API!!

Answer 6

Hello all. Have you made any progress with your facebook crawler? I am doing some research in social networks and I want to be able to get some data from a group that I am an admin. Mostly collect daily how many users are added, and their connection with existing members.