Screen scraping through AJAX and javascript

Question

Hi friends,

I want to scrape a screen using AJAX and javascript. How is this possible?

Need to scrape this : http://www.fedex.com/Tracking?ascend_header=1&clienttype=dotcom&cntry_code=us&language=english&tracknumbers=776812461212825

I had used the technique given in w3school.com, but it's showing "access denied." Why is the "access denied" showing? Please solve my problem..

Thanks in advance.

Answer 1

If you are screen scraping by trying to access the FedEx URL directly from your Javascript/website on your own domain, that is your problem. If you really want to do this, the easiest way would be to make an AJAX call to a script somewhere in your domain and have it make the tracking request for you.

Check out this information on creating proxies.

Answer 2

Ajax/XMLHttpRequest calls are restricted to a same-site policy for security reasons; you can't use them to directly load remote sites. Firefox 3.1 apparently added support for cross-site requests, but I haven't heard if any other browsers jumped.

Answer 3

Building on Beau Simensen answer, you could make an Ajax call to your own domain:

Ajax.Request("http://mydomain.com/proxy.php?url=http://www.fedex.com/Tracking?ascend_header=1&clienttype=dotcom&cntry_code=us&language=english&tracknumbers=776812461212825");

The proxy.php file would just make a CURL request (or something else) to the url parameter provided and return its result, this way you can access all sites on your local domain through that proxy.php.