views:

166

answers:

4

I'm building a database + tool that scours news feeds for a certain term. For example "food poisoning from nuts". I want to scour social media sites, news sites, major news aggregators, etc... for that term.

  • Question 1: What are some of the news aggregator APIs out there?
  • Question 2: How Would you go about coding and receiving only the latest news from the API?

Edit Added schematic: alt text

A: 

Google has a news feeds api but I think it is only for non commercial use http://www.google.com/support/news/bin/answer.py?answer=59255&hl=en

anijhaw
A: 

If you've got monetary support backing your project, Reuters has news data feeds that you can subscribe to on a monthly basis.

Dr. Watson
+2  A: 

Do you know Yahoo! Pipes? It's a very flexible feed aggregator, and you can manipulate it using YQL, which is quite powerful and has a Python librabry, python-yql :). YQL also has specific "tables" for Twitter and other services and news sources, so, depending on what you want, you might not even need Pipes.

There's a quick example of Pipes + YQL usage at http://blog.ouseful.info/2009/04/27/using-yql-with-yahoo-pipes/ , but you can play around with them at the YQL Console.

So, that's your "Question 1". But, using YQL, "Question 2" (if I understood it correctly) comes naturally, as you use it almost like regular SQL, imposing limits, ordering etc.

rbp
@rbp, +1 I was looking for something like this :)Here is an interesting [slide](http://www.slideshare.net/ConSanFrancisco123/yahoo-pipes-middleware-in-the-cloud)
Sandy
@Sandy: nice, good reference :)
rbp
+1  A: 

Have you tried Universtal Feed Parser ??

Maddy
A quick glance on the website says that they haven't been updated since 2007
dassouki
But that doesn't do what you want ?
Maddy