get web page text via javascript

Question

Is there a JavaScript statement that will retrieve the contents/text from a web page?

Answer 1

That's kind of a weird thing to do. Do you mean anything between tags like <p> and <span> or something more sinister like <title> as well? Are you sure you need to do this in JS if you're scraping a whole page? Sounds like an XSS attack :P

Offhand, I don't know of anything builtin that will do that, but you can always just write a function to access the DOM with document.getElementsByTagName and scrape the text that way.

Answer 2

You could do it with Ranges / TextRanges. This has the advantage of only getting the visible text on the page (unlike, for example, the textContent property of elements in non-IE browsers, which will also get you the contents of <script> and possibly other elements). The following will work in all mainstream browsers although I can't make any guarantees about the consistency of line breaks between different browsers.

function getBodyText(win) {
    var doc = win.document, body = doc.body, selection, range, bodyText;
    if (body.createTextRange) {
        return body.createTextRange().text;
    } else if (win.getSelection) {
        selection = win.getSelection();
        range = doc.createRange();
        range.selectNodeContents(body);
        selection.addRange(range);
        bodyText = selection.toString();
        selection.removeAllRanges();
        return bodyText;
    }
}

alert( getBodyText(window) );

Answer 3

If you just want the HTML content of the page you're currently on:

var html = document.getElementsByTagName('html')[0].innerHTML;

If you want the HTML for another page Google "Ajax"

Answer 4

If the page you want to scrape comes from same domain, it's fairly straight forward. You just use AJAx to get that page as text and then find what you want from there. Here's an example:

<script>
  function ajax(url, callbackFunction) {
    var request =  new XMLHttpRequest();
    request.open("GET", url, true);
    request.setRequestHeader("Content-Type",
      "application/x-www-form-urlencoded");

    request.onreadystatechange = function() {
      var done = 4, ok = 200;
      if (request.readyState == done && request.status == ok) {
        if (request.responseText) {
          callbackFunction(request.responseText);
        }
      }
    };
    request.send();
  }

  var buttonClicked = function () {
    ajax('/~jedi/index.html', getComplete);
  }

  var getComplete = function (text) {
    alert(text);
  }
</script>

<button onclick="buttonClicked()">Debug</button>

If the page comes from a different domain you are out of luck. You would need to do that on the server. You are able to load data on your page from a different domain using an IFrame (you can create an invisible IFrame programmatically) but the browser won't allow you to read that data due to security policies.