PubDate/Guid is essential to RSS? How I create a good RSS in Yahoo! Pipes if the source doesn't provide different dates for the items?

Question

I am creating a Yahoo! Pipe to a news site but the feedless source doesn't have a date/time for each item. My RSS doesn't works very well: each update makes the RSS Reader, Google Reader for instance, to mark all readed items as unreaded again. Perhaps that's because of the lack of pubDate tag or incorrect guid tag.

1) How to create a "pubDate" on Yahoo! Pipes when your source doesn't provide you the data?

2) How to avoid the "guid" tag overwritting? (you can set the guid in YPipes but then YPipes ignores your guid)

Solution: pudDate isn't necessary. guid is essential. Even if Yahoo! Pipes rewrites the guid, it will work, because Yahoo! Pipes converts your guid text into a hash value, that do not is modified until the text is modified.

Answer 1

I don't have a definitive answer for you, but anecdotely I have been maintaining a private feed reader for the last 4 years or so. I've been exposed to a lot of vagaries of RSS/ATOM and I can tell you that a lot of feeds don't have dates associated with the items. It might be an RSS version issue.

Answer 2

the problem could be with the source of your feed. If you are using mutliple feeds then after the union operation in pipes, do a sort operation on pubdate and then redirect it to the output.

Answer 3

Last time I rebuilt my site, I had a bunch of trouble with the feed. In the way you describe- read things becoming unread on next update, duplicate entries. Turns out the problem was more to do with the guid element than the pubdate. As far as I recall, it didn't matter too much what I did with the date (I had the format wrong for a while) as long as the guid was unique.

Answer 4

Have you looked at Feedity - http://feedity.com - for creating custom RSS feeds. It's like Pipes, but much easier to use, and in fact works well within Pipes as well. I've been using it for a while to create RSS feeds for those "feedless" webpages.

Answer 5

I think the GUID is generated from the link parameter. So it is important to have a unique url for each feed item. If all the feed urls have same link, they will have same GUID.

I hope that helps.

I am struggling myself to create unique url. Have you found anyway to achieve it?

Answer 6

With Yahoo Pipes, using the 'Create RSS' module, it appears to use (a hashed version of) each entry's link to generate a GUID, which as you point out, is necessary for most feed readers to detect new entries.

I've attempted to set the 'Create RSS' module's GUID field to a value that's unique for each entry, however the GUID in the resultant feed remains identical for each entry. When I then set the link to this value the GUIDs generate were unique for each entry.

I have verified this by making a copy of your pipe and removing (well, renaming) the link attribute and no GUID is generated (although you have specified one). This has been confirmed by others as a bug, see tinyurl.com/mxard2.

Answer 7

Just been doing this myself, and have resorted to appending a random number to the url that I'm using to get the data from (I'm scraping using YQL). I'm generating that random number by using a Date Builder and populating it with "today" to get the current date/time. I'm then using a URL Builder to build up my url that I'm requesting, passing in an extra parameter of "randomnumber" which I'm assigning to my the DateTime.utime value.

Having looked at the generated RSS feed via view source, the articleId now does appear to be unique, but I haven't left it long enough to know if google reader etc sees it as different.