tags:

views:

145

answers:

3
+1  Q: 

Backlink-reporting website crawler?

What tools are there out there to crawl a website and report, for each page, a list of pages within the website that link to it?

A: 

W3C has a good link checker at: http://validator.w3.org/checklink.

Alan Haggai Alavi  2009-05-25 01:41:55
Oh, and search the page for instances of whatever URL I want to check. I guess it'll do for now ... at least, for as long as my site isn't going over the 150-page limit.
Stewart  2009-05-27 14:11:21
A: 

ruby's mechanize lets you load a page and then extracts all its links using a single function call

adi92  2009-05-25 02:08:24
A: 

http://arachnode.net can do this easily - uses SQL for storage.

arachnode dot net  2009-08-29 09:31:05

related questions

Combined SVN FTP system?
Looking for ways to automate web site testing
Find coordinates of every link in a page
What is your preferred site for code snippets
Javascript make the web broswer srcoll to the top?
Best Ruby on Rails social networking framework
Twitching Consumption of Web Services from Web Site to Web Application
How to use chrome to login to same site twice with different credentials ?
Looking for a Good Resource for Print Designers that are doing Web Design
How does dedicated webhosting compare to Amazon's Cloud?
What makes a good website for an open source project?
Communicating between websites (using Javascript or ?)
Which CMS or other framework should I use to develop a stackoverflow-like site?
How to store Application Messages for a .NET Website
What to charge for a simple website
What is the best solution for maintaining backup and revision control on live websites?
What is a good Content Management System to deploy for a personal website/blog?
What do most users consider acceptable load times on a website?
Should my website abandon username/password authentication in favor of OpenID?
Shopping Cart
Personal Website Construction
HTTPS in IIS 5.1
Do you think using websites like Elance, to obtain side jobs while still working full time, could help my programming career?
Whatever happened to VB Accelerator?
My website got hacked... What should I do?