screen scrapping business contact details, legal?

Question

I was wondering if it was legal, in the UK, to do this.

Basically there are hundreds of websites that just display contact details of businesses, like online directories.

If I were to scrape these kinds of pages for the details to put on a different directory site would I be commiting a crime?

I was thinking of using HtmlAgilityPack to get the details. I believe this tool is legal as it has been recomended on this site before.

I think actually running a program to get it would be ok as it could be used for offline personal use?

Also as noted before I could have just got this info elsewhere anyway so it's not like original written content etc.

I just don't want to be doing anything illegal though.

Answer 1

StackOverflow is not a site that to solicit legal advice. Scraping is not illegal, but putting data collected from it into commercial use, may not be the best strategy for various reasons.

Answer 2

Its better to ask the site owners as the terms and conditions may differ from site to site.

Check out folowing related questions.

http://stackoverflow.com/questions/822380/how-legal-is-screen-scraping

http://stackoverflow.com/questions/396778/legalities-of-screen-scraping

Answer 3

The site i was looking at never had any links to terms of use or anything similar.

Answer 4

Sites can have a robots.txt file set up which is used to block access to certain search engines. They use this because web crawling will eat up their bandwidth which is a waste of time and money for many companies. Thus they use a robots.txt to limit those visitors. It tells search engines exactly where they can and cannot crawl into.

This site also has such a file, which you can find at http://stackoverflow.com/robots.txt

If you respect these settings, you should be just fine.

However, there is another legal aspect that you need to consider. No matter how you collect this information from these other sites, if you display them on your own site you might get in trouble over privacy issues or copyright violations. This means you can still scrape those sites and use the information for personal reasons, but if you republish them, you might end up in some nasty legal troubles. So before you publish data from other sites, ask those site owners for permission to distribute that information.

Answer 5

StackOverflow is uniquely unsuited to providing legal advice. However, you should be aware of so-called database rights (since you are in the UK). Whether you are doing something for personal use or for commercial purposes usually doesn't affect its legality, but will have an impact on how many people are likely to want to come after you. And remember - just because you haven't done anything wrong doesn't mean they won't sue you.