What's the best way of extracting javascript links in an HTML document?

Question

Hi All,

I am writing a small webspider for a website which uses a lot of javascript for links:

<htmlTag onclick="someFunction();">Click here</htmlTag>

where the function looks like:

function someFunction() {
  var _url;
  ...
  // _url constructed, maybe with reference to a value in the HTML doc
  // and/or a value passed as argument(s) to this function
  ...
  window.location.href = _url;
}

What is the best way of evaluating this function server-side so I can construct the value for _url?

Answer 1

Not exactly sure what you're trying to accomplish.

If you need to send these values to the server for processing, Ajax would be your best option.

Answer 2

It should be a mess to do. But it depends on a lot of params:

1. Where does the link is stored ? inside the element, in a javascript var, etc...
2. Is the javascript function always be your own ?

Some hints that could do the trick, should to simply parse your html and use regex to catch http links where the onclick="someFunction();" attribute is present.

Answer 3

If you need server-side processing, you need to either:

1. Do the processing before the content is delivered to the user, and include its output in the response, or
2. Use something like AJAX to make a new request back to the server

Answer 4

You could also use env.js and rhino to actually evaluate the JavaScript in the html and detect changes to the location object after manually firing a click event.