views:

772

answers:

3

I've used Yahoo Pipes for some prototyping, but I'm unwilling reluctant to use it in full production as we'll have no control over it uptime etc.

Is there an (ideally open source) alternative that I can run on my own server? Or any application that can convert a pipe into PHP or Ruby?

+1  A: 

In our company, we've been using Yahoo Pipes for quite a long time and we're still using it for some specific and more complex feeds.

Otherwise, we are developing a simple Sinatra-based application to work like Yahoo Pipes and return the result as a standardized XML/JSON response. It's quite simple to create such application, however I agree with Pascal. Between Yahoo's servers and yours, relying on Yahoo! should be more effective than relying on a custom server.

The only Pipes downside we've been experiencing is the API throttle. We've partially solved the issue caching the responses on our side and implementing a custom throttling mechanism to dynamically slow down requests when Pipes returns an HTTP error.

Simone Carletti
Good to hear a success story. But it would be great to have an open source alternative that you could write your own modules for.
rjstelling
A: 

Regarding uptime, you could cache the output of Yahoo Pipes on your server in case the feed goes AWOL for a short time (though as PascalMartin said, their uptime is very likely better than yours).

It's unlikely that Yahoo would suddenly close Yahoo Pipes but if they did I'm sure clones would start springing up. Worst case scenario it wouldn't be too difficult to write a custom program/script to fetch a bunch of feeds and filter them on your criteria.

DisgruntledGoat
+1  A: 

You can generate a Python code equivalent of a simple Yahoo pipe with pipe2py: http://github.com/ggaughan/pipe2py

If a generated code equivalent of Yahoo Pipes block you require isn't part of the distribution, pipe2py is easily extended/ (Please consider contributing any extensions back...)

psychemedia