views:

2440

answers:

3

Hello ll,

I know JavaScript can open a link in a new window but is it possible to open a webpage without opening it in a window or displaying it to the user? What I want to do is parse that webpage for some text and use it as variables.

Is this possible without any help from server side languages? If so, please send me in a direction I can achieve this.

Thanks all

+6  A: 

You can use an XMLHttpRequest object to do this. Here's a simple example

var req = new XMLHttpRequest();  
req.open('GET', 'http://www.mydomain.com/', false);   
req.send(null);  
if(req.status == 200)  
   dump(req.responseText);

Once loaded, you can perform your parsing/scraping by using javascript regular expressions on the req.responseText member.

More detail...

In practice you need to do a little more to get the XMLHttpRequest object in a cross platform manner, e.g.:

var ua = navigator.userAgent.toLowerCase();
if (!window.ActiveXObject)
  req = new XMLHttpRequest();
else if (ua.indexOf('msie 5') == -1)
  req = new ActiveXObject("Msxml2.XMLHTTP");
else
  req = new ActiveXObject("Microsoft.XMLHTTP");

Or use a library...

Alternatively, you can save yourself all the bother and just use a library like jQuery or Prototype to take care of this for you.

Same-origin policy may bite you though...

Note that due to the same-origin policy, the page you request must be from the same domain as the page making the request. If you want to request a remote page, you will have to proxy that via a server side script.

Another possible workaround is to use Flash to make the request, which does allow cross-domain requests if the target site grants permission with a suitably configured crossdomain.xml file.

Here's a nice article on the subject of the same-origin policy:

Paul Dixon
Damn! I wwas hoping for an external page, so because of CSC JavaScript is not allowed to parse text from an external web page?
Abs
Not so much parsing but the XMLHttpRequest is forbidden from fetching from another domain. The workaround is to request via your own server-side script, or you could try using a small iframe.
Paul Dixon
I understand now. It will have to be a server side implementation then.
Abs
Having said that, I know Flash can let you do cross domain requests as long as the target site is configured to allow them. Will add to answer
Paul Dixon
Ah thanks. Flash is dangerous territory for me though!
Abs
+3  A: 

You could open the new window in an iframe:

http://www.w3schools.com/TAGS/tag_iframe.asp

Although note that Javascript access is limited if the site you open is from a different URL. This is to prevent cross-site scripting attacks:

http://en.wikipedia.org/wiki/Cross-site_scripting

Chris
+1 this is what I'd do if the site isn't mine and doesn't support pjson callbacks
Robert Gould
+2  A: 

You would use AJAX. This would make a Get request to the URL in question and return the response HTML. Jquery makes this very easy e.g.

$.get("test.php");

http://docs.jquery.com/Ajax

Andrew

REA_ANDREW