I'm trying to read and handle a web-page in Python which has lines like the following in it:
<div class="or_q_tagcloud" id="tag1611"></div></td></tr><tr><td class="or_q_artist"><a title="[Artist916]" href="http://rateyourmusic.com/artist/ac_dc" class="artist">AC/DC</a></td><td class="or_q_album"><a title="[Album374717]" href="http://rateyourmusic.com/release/album/ac_dc/live_f5/" class="album">Live</a></td><td class="or_q_rating" id="rating374717">4.0</td><td class="or_q_ownership" id="ownership374717">CD</td><td class="or_q_tags_td">
I'm currently only interested in the artist name (AC/DC) and album name (Live). I can read and print them with libxml2dom but I can't figure out how I can distinguish between the links because the node value for every link is None.
One obvious way would be to read the input line at a time but is there a more clever way of handling this html file so that I can create either two separate lists where each index matches the other or a struct with this info?
import urllib
import sgmllib
import libxml2dom
def collect_text(node):
"A function which collects text inside 'node', returning that text."
s = ""
for child_node in node.childNodes:
if child_node.nodeType == child_node.TEXT_NODE:
s += child_node.nodeValue
else:
s += collect_text(child_node)
return s
f = urllib.urlopen("/home/x/Documents/rym_list.html")
s = f.read()
doc = libxml2dom.parseString(s, html=1)
links = doc.getElementsByTagName("a")
for link in links:
print "--\nNode " , artist.childNodes
if artist.localName == "artist":
print "artist"
print collect_text(artist).encode('utf-8')
f.close()