Malicious crawler blocker for ASP.NET

Question

I have just stumbled upon Bad Behavior - a plugin for PHP that promises to detect spam and malicious crawlers by preventing them from accessing the site at all.

Does something similar exist for ASP.NET and ASP.NET MVC?

I am interested in blocking access to the site altogether, not in detecting spam after it was posted.

EDIT: I am interested specifically in solutions that will detect access patterns to the site - these would prevent screen scraping the site as a whole, or at least make it a very slow process for the offender because the bot would have to act as a regular user in frequency of his clicks and in patterns found in the clicks (100 links visited in depth-first succession are a clear indication that someone is crawling the site).

By malicious crawler I mean primarily bots that are building duplicate copies of other sites but also spambots that are trying to locate some submittable forms on the site and submit their crap.

Answer 1

www.atlbl.com do an API that can detect bad webcrawlers/spammers/etc which you can use to block access to specific URL's or your entire site.

Answer 2

The NoBot control in the ASP.NET AJAX Toolkit might help, if I understand you correctly. From the description page:

NoBot is a control that attempts to provide CAPTCHA-like bot/spam prevention without requiring any user interaction. This approach is easier to bypass than an implementation that requires actual human intervention, but NoBot has the benefit of being completely invisible.

I haven't used this one myself, so I can't speak to its effectiveness against sophisticated bots, but it sounds like it's meant to be exactly what you're talking about. It has the benefit of being very simple; essentially a drop-in you can add to a master page.

Answer 3

I am not sure if it is worth the effort (and false postives with real users!). Adding random delays to an iMacros or Perl script is a "trick" something every bot programmer knows.

Instead, I would either require a login or display captchas.