views:

40

answers:

0

Is there any robots.txt parser written in JavaScript? I'm usually coding with python and there's robotparser: http://docs.python.org/library/robotparser.html, which is very easy to use.