How can I iterate over all tags which have a specific attribute with a specific value? For instance, let's say we need the data1, data2 etc... only.
<html>
<body>
<invalid html here/>
<dont care> ... </dont care>
<invalid html here too/>
<interesting attrib1="naah, it is not this"> ... </interesting tag>
<interesting attrib1="yes, this is what we want">
<group>
<line>
data
</line>
</group>
<group>
<line>
data1
<line>
</group>
<group>
<line>
data2
<line>
</group>
</interesting>
</body>
</html>
I tried BeautifulSoup but it can't parse the file. lxml's parser, though, seems to work:
broken_html = get_sanitized_data(SITE)
parser = etree.HTMLParser()
tree = etree.parse(StringIO(broken_html), parser)
result = etree.tostring(tree.getroot(), pretty_print=True, method="html")
print(result)
I am not familiar with its API, and I could not figure out how to use either getiterator or xpath.