Crawling, spidering, getting data

Question

Hi All,

I want to write a crawler to do some inventory of infrastructure (routers, WAPS, systems, etc)

So I understand a crawler, implementing is a different story.

So I have an app sitting on a server and how do I deploy it. How does the crawler start looking? Say I wanted to inventory unknown subnets, etc?

How does the crawler send back data?

Thoughts on this are appreciated.

Answer 1

Just one program that will do what you're talking about is nmap from insecure.org (author is fyodor if I remember correctly.) Works on multiple platforms. It might be useful to start with one of the dozens to hundreds of available network and infrastructure scanners out there to get an idea of what you can do, and how to do it.