How to offer a website scrapping service like dapper.com, diffbot.com? (especially the visual selection)

Question

Hi,

I'm trying to build a functionality for the users of my site to "screen scrap" websites of their choice and I'm looking for a toolkit that allows me to do that OR a third party site that allows other sites (like mine) to use their interface.

Let me try explaining it better. I have customers on my site. These customers are interested in monitoring other websites and keeping track of changes and updates on these websites. I am interested in providing them an interface so that they can visually select what they want to monitor / track. Then my site (or through the 3rd party service) periodically monitors those webpages and my site acts on it when changes are detected.

So I'm looking for a toolkit which can help build the great visual selection portion that these sites have (dapper.com, diffbot.com) OR a third-party service that allows me to do the same.

Thanks!
Steve

Answer 1

This is a very hard project to get done right and I don't know of any available 3rd party service.

I had a go once by loading the desired webpage and then adding click events that allowed the user to select any text region. It worked, but was very clunky!