free RSS feed caching

Question

Hello

I've got an application which serves an rss feed of headlines and I need to provide this rss feed to other consumers. I don't want to provide the rss directly from my server though, due to limited server resources (processing and also bandwidth), so I need to proxy (cache) it through some service which will handle most of the traffic.

Assuming the rss feed URL of my application is http://example.com/rss I initially provided my consumers with the url http://ajax.googleapis.com/ajax/services/feed/load?v=1.0&q=http%3A%2F%2Fexample.com%2Frss which solved my server load problem but introduced a liveness problem. The headlines are minutes to hours late from the actual feed (haven't exactly measured how much late).

I've also tried distributing through feedburner so the url became something like http://feeds.feedburner.com/example123?format=xml but the liveness problem still exists.

Is there a public and free solution for this problem? Anything below 5 minutes of liveness delay would be totally acceptable.

thanks

Answer 1

Have you tried Yahoo Pipes? It saved a colleague of mine from a few headaches, the RSS feed output from a pipe was refreshed roughly every 30 mins IIRC.

Answer 2

Have you considered installing a cache yourself?

You don't give many details of how your feed is served, but I've had good success in the past installing Apache to listen on port 8080, and then having a small, fast, and efficient webserver like nginx running on port 80.

The idea is that nginx is very very fast at serving static files, and will do that very quickly. Anything else will be passed to the local Apache instance(s) and your server load will probably fall down - despite having two webservers running.

Answer 3

The way I solved it is by using the feedburner feed and pinging it (at http://www.feedburner.com/fb/a/pingSubmit?bloglink=http%3A%2F%2Fexample.com) via wget every 5 minutes with a cron job in order to update the contents of the feed. Works great!