How to extract url GET parameter from <a> tag, from the full html text.

Question

So I have an html page. It's full of various tags, most of them have sessionid GET parameter in their href attribute. Example:

...
<a href="struct_view_distrib.asp?sessionid=11692390">
...
<a href="SHOW_PARENT.asp?sessionid=11692390">
...
<a href="nakl_view.asp?sessionid=11692390">
...
<a href="move_sum_to_7300001.asp?sessionid=11692390&mode_id=0">
...

So, as you see, sessionid is the same, i just need to get it's value into variable, no matter from which one: x=11692390 I'm newbie in regex, but google wasn't helpful. Thx a lot!

Answer 1

Parse your HTML with a DOM parsing library and use getElementsByTagName('a') to grab anchors, iterate through them and use getAttribute('href') and then extract the string. Then you can use regex or split on ? to match/retrieve the session id.

Answer 2

Below is an regex you can use to match hrefs and extract its value:

\b(?<=(href="))[^"]*?(?=")

Answer 3

I would do this - before I was told it was a python issue ;)

<script>
function parseQString(loc) {
  var qs = new Array();
  loc = (loc == null) ? location.search.substring(1):loc.split('?')[1];
  if (loc) {
    var parms = loc.split('&');
    for (var i=0;i<parms.length;i++) {
      nameValue = parms[i].split('=');
      qs[nameValue[0]]=(nameValue.length == 2)? unescape(nameValue[1].replace(/\+/g,' ')):null; // use null or ""
    }
  }
  return qs;
}
var ids = []; // will hold the IDs
window.onload=function() {
  var links = document.links;
  var id;
  for (var i=0, n=links.length;i<n;i++) {
    ids[i] = parseQString(links[i].href)["sessionid"];
  }
  alert(ids); // remove this when happy
  // here you can do 
  alert(ids[3]); 
  //to get the 4th link's sessionid
}


</script>

<a href="struct_view_distrib.asp?sessionid=11692390">
...</a>
<a href="SHOW_PARENT.asp?sessionid=11692390">
...</a>
<a href="nakl_view.asp?sessionid=11692390">
...</a>
<a href="move_sum_to_7300001.asp?sessionid=11692390&mode_id=0">
...</a>

Answer 4

This does not use regexes, but anyway, this is what you would do in Python 2.6:

from BeautifulSoup import BeautifulSoup
import urlparse

soup = BeautifulSoup(html)
links = soup.findAll('a', href=True)

for link in links:
  href = link['href']
  url = urlparse.urlparse(href)
  params = urlparse.parse_qs(url.query)
  if 'sessionid' in params:
    print params['sessionid'][0]