tags:

views:

116

answers:

3
+1  Q: 

Scraping and web crawling framework, PHP

Hi,

I know scrapy.org that is a fast high-level screen scraping and web crawling framework, used to crawl websites and extract structured data from their pages. I used it in some projects and it is very simple to use. But it is written in python.

My question is, are there simlar frameworks for php?

+3  A: 

Snoopy

Galen  2010-10-07 20:54:08
thanks, but can not find examples, is it possible to extract data from specific html elements? E.g. give it a pattern to extract all data located in html/body/div/
ArtWorkAD  2010-10-07 21:00:33
A: 

You may find SimpleHTMLDom http://simplehtmldom.sourceforge.net/ with cURL is enough for your needs.

D Roddis  2010-10-07 21:24:04
Suggested third party alternatives to [SimpleHtmlDom](http://simplehtmldom.sourceforge.net/) that actually use [DOM](http://php.net/manual/en/book.dom.php) instead of String Parsing: [phpQuery](http://code.google.com/p/phpquery/), [Zend_Dom](http://framework.zend.com/manual/en/zend.dom.html), [QueryPath](http://querypath.org/) and [FluentDom](http://www.fluentdom.org).
Gordon  2010-10-07 21:26:19
don't use simpleHtmlDom. it is slow and offers less functionality than the built in dom libs.
Byron Whitlock  2010-10-07 21:54:13
+1  A: 

You may find ScraperWiki interesting. It has tutorials on various scraping topics and an online generator for scraper scripts in PHP, Python and other languages.

Gordon  2010-10-07 21:31:25

related questions

IDE suggestions: Eclipse IDE vs. Zend Studio ( confused )
MySQL/Apache Error in PHP MySQL query
Lightweight IDE for Linux
What PHP framework would you choose for a new application and why?
Why is my ternary expression not working?
How can I get at the matches when using preg_replace in PHP?
Mechanisms for tracking DB schema changes
Wordpress theme development offline tools
Using object property as default for method property
How can I get the authenticated user name under Apache using plain HTTP authentication and PHP?
Make XAMPP/Apache serve file outside of htdocs
How do you debug PHP scripts?
PHP Variables passed by value or by reference?
Best way to implement unit testing in PHP
Connect PHP to an AS/400
Best way to access Exchange using PHP?
PHP Session Security
How do I access a remote form in php?
What's the best way to generate a tag cloud from an array? (using h1 through h6 for sizing)
Apache/PHP: error_log per Virtual Host?
How do I track file downloads with apache/PHP
How would you access Object properties from within an object method?
Flat File Databases in PHP
Best way to allow plugins for a PHP application
Latest information on PHP upcoming releases