views:

84

answers:

3

How do you practice web scraping (for example, authentication) when you're learning? Do you:

  1. Practice with your real username and password on a real web site, and hope you don't mess up (eg fail authentication too many times, accidentally hammer the web site)?
  2. Create a fake username and account for this purpose, and hope they don't keep track of IP addresses?
  3. Create your own web site and practice authentication against that?

Or is there a sandbox web site where you can practice web scraping without any reprecussions?

+1  A: 

3 is the best option, if available to you.

That way you don't risk getting shut down by any third party you are attempting to scrape.

Oded
+1  A: 

I will use 1. But just need to validate each scraping step before running the whole process.

seagulf
A: 

(1) is risky and (3) takes a lot more time. I use (2) with proxy addresses to avoid being blocked.

Plumo