Assuming I have html read into my program like this:

<p><a href=""&gt;F/T &amp; P/T Sales Associate - Caliente Fashions</a> - <font size="-1"> (North Vancouver)</font></p>
<p><a href=""&gt;IMMEDIATE EMPLOYMENT WANTED!</a> - </p>

<p><a href=""&gt;TRAVEL AGENT</a> - <font size="-1"> (NORTH VANCOUVER)</font></p>
<p><a href=""&gt;Optical Sales Position</a> - <font size="-1"> (New Westminster)</font></p>
<p><a href=""&gt;Sales Clerk</a> - <font size="-1"> (Kits)</font></p>

<p><a href=""&gt;MARINE SALES</a> - <font size="-1"> (VANCOUVER ( KITS ))</font></p>
<p><a href=""&gt;Retail Sales Associate</a> - <font size="-1"> (Vancouver)</font></p>
<p><a href=""&gt;Retail with small parts appliance background</a> - </p>
<p><a href=""&gt;Manager *Enjoyable work atmosphere</a> - <font size="-1"> (Langley Centre)</font></p>

<p><a href=""&gt;Team Member - Retail Store - FT</a> - <font size="-1"> (Burnaby South)</font></p>
<p><a href=""&gt;STORE MANAGER-SHOE WAREHOUSE</a> - <font size="-1"> (South Surrey-Semiahmoo)</font></p>
<p><a href=""&gt;Retail Sales</a> - <font size="-1"> (Coquitlam)</font></p>

How do I grab the contents of the text node? What I would like to end up with is printing something similar to this line in the terminal: - TRAVEL AGENT

So far I have the following code which extracts the href link fine but I'm not sure how to extract the data itself. I'm thinking of overriding handle_data(self, data) from the module but so far I can't seem to think of a way to do it.

from sgmllib import SGMLParser

class URLLister(SGMLParser):
    def reset(self):
        self.urls = []

    def start_a(self, attrs):
        href = [v for k, v in attrs if k == "href"]
        if href:


+3  A: 

Personally I would use lxml. Once installed, getting what you want is simple:

from lxml import html

tree = html.fromstring(open("data.html").read())

print [e.text_content() for e in tree.xpath("//a")]
+5  A: 

Simplest is probably BeautifulSoup (be sure to use 3.0.8 or higher 3.0.* release, not 3.1.*, unless you're on Python 3 -- see here!).

import BeautifulSoup
soup = BeautifulSoup.BeautifulSoup(thehtmlstring)

for anchor in soup.findAll('a'):
  print a['href'], a.string

BeautifulSoup produce unicode strings -- if that's a problem, be sure to encode them as you wish to get the byte strings the way you want them!

Alex Martelli
If following this REALLY DON'T USE 3.1.* (I should read everything before diving in to it) :)
Diego Castro
+2  A: 
I like: `for k, v in attrs: if k == 'href'; return v`
Good call. I updated my post.
+1  A: 

As long as we're comparing options, this pyparsing snippet also gives you the location for each position, given in the <font> tag following the closing <a> tag:

from pyparsing import makeHTMLTags, SkipTo

a,aEnd = makeHTMLTags("A")
font,fontEnd = makeHTMLTags("FONT")
p,pEnd = makeHTMLTags("P")

patt = (p + a("a") + SkipTo(aEnd)("posn") + aEnd + '-' + 
        font + SkipTo(fontEnd)("locn") + fontEnd + pEnd)

for tokens,_,_ in patt.scanString(the_html):
    print tokens.a.href, '-', tokens.posn, tokens.locn

Gives: - F/T &amp; P/T Sales Associate - Caliente Fashions (North Vancouver) - TRAVEL AGENT (NORTH VANCOUVER) - Optical Sales Position (New Westminster) - Sales Clerk (Kits) - MARINE SALES (VANCOUVER ( KITS )) - Retail Sales Associate (Vancouver) - Manager *Enjoyable work atmosphere (Langley Centre) - Team Member - Retail Store - FT (Burnaby South) - STORE MANAGER-SHOE WAREHOUSE (South Surrey-Semiahmoo) - Retail Sales (Coquitlam)
Paul McGuire