Can I programatically open, search and return results in a browser from multiple websites?

Question

Currently I have a bookmarked list of websites that I look for content on, and open them in a new window, in tabs, and search for the string that I want to find.

Is there any way to do this programatically, ending up with a browser window containing the results pages of the sites? There's around 20 of them.

I'm running Safari 3 on Mac OS X Leopard. I have experience using PHP, but even something as simple as using AppleScript would be OK too. I just want to speed up my workflow. :-)

Answer 1

You are going to have to specify which browser you are using, because I am sure that each one has different automation capabilities.

Answer 2

Well if you know ruby, there is ScRUBYt!

http://www.softwaredeveloper.com/features/scrubyt-ruby-web-scraping-tool-051007/
http://scrubyt.org/

Answer 3

Here's are a couple of articles about writing a web crawler in perl:

• http://www.linuxjournal.com/article/2200
• http://www.devshed.com/c/a/Perl/Web-Mining-with-Perl/

Answer 4

Not entirely programming related, but you could create something like this using Google's Custom Search Engine facility (assuming all the sites you plan to search are publicly accessible).

You can then insert the provided snippets into an HTML file of your choice, and even set this as your "Home Page" in Firefox.

Answer 5

Try Yahoo Pipes, there is probably one that's alredy half way where you want it.

I searched it for "search" and here are resoults.

http://pipes.yahoo.com/pipes/search?q=search&x=0&y=0