What is the best way to get RSS Feeds into a MySQL Database

Question

I am trying to take several RSS feeds, and put the content of them into a MySQL Database using PHP. After I store this content, I will display on my own page, and also combine the content into one single RSS Feed. (Probably after filtering)

I haven't dealt with RSS Feeds before, so I am wondering the best Framework/Method of doing this is. I have read about DOM based parsing, but have heard that it takes a lot of memory, any suggestions?

Answer 1

Magpie is a reasonable RSS parser for PHP. Easy to use:

require('rss_fetch.inc');
$rss = fetch_rss($url);

An item like this for example:

<item rdf:about="http://protest.net/NorthEast/calendrome.cgi?span=event&amp;ID=210257"&gt;
<title>Weekly Peace Vigil</title>
<link>http://protest.net/NorthEast/calendrome.cgi?span=event&amp;ID=210257&lt;/link&gt;
<description>Wear a white ribbon</description>
<dc:subject>Peace</dc:subject>
<ev:startdate>2002-06-01T11:00:00</ev:startdate>
<ev:location>Northampton, MA</ev:location>
<ev:enddate>2002-06-01T12:00:00</ev:enddate>
<ev:type>Protest</ev:type>
</item>

Would be turned into an array like this:

array(
    title => 'Weekly Peace Vigil',
    link => 'http://protest.net/NorthEast/calendrome.cgi?span=event&ID=210257',
    description => 'Wear a white ribbon',
    dc => array (
      subject => 'Peace'
     ),
    ev => array (
     startdate => '2002-06-01T11:00:00',
     enddate => '2002-06-01T12:00:00',
     type => 'Protest',
     location => 'Northampton, MA'
    )
);

Then you can just pick out the bits you want to save in the DB and away you go!

Answer 2

There are several RSS parsing libraries out there, including Magpie and one in pear.

I would pick a parser, and then run it through a loop with the data to feed it into the database. Make sure you figure out how often you want to run the script, and think about if this is running from cron, or part of a page that is only loaded infrequently.

Answer 3

The Best PHP parser out there is SimplePie, IMHO. I've been using it for years. It's great at grabbing and parsing the following: RSS 0.90, RSS 0.91 (Netscape), RSS 0.91 (Userland), RSS 0.92, RSS 1.0, RSS 2.0, Atom 0.3, Atom 1.0; including the following namespaces: Dublin Core 1.0, Dublin Core 1.1, GeoRSS, iTunes RSS 1.0 (mostly complete), Media RSS 1.1.1, RSS 1.0 Content Module, W3C WGS84 Basic Geo, XML 1.0, XHTML 1.0

SimplePie 1.2 even has database caching, so it should have everything you need to do what you want.

And if you need to parse raw XML files, try using XMLize

-Trystian

Answer 4

For a very simple hacked together script that just works end-to-end (parse RSS, insert into DB);

http://code.google.com/p/rssingest/