How do I query XHTML using python?

Question

I have created a simple test harness in python for my ASP .net web site.

I would like to look up some HTML tags in the resulting page to find certain values.\

What would be the best way of doing this in python?

eg (returned page):

<div id="ErrorPanel">An error occurred......</div>

would display (in std out from python):

Error: .....

or

<td id="dob">23/3/1985</td>

would display:

Date of birth: 23/3/1985

Answer 1

Do you want to parse XML, as you state in your question's title, or HTML, as you show in the text of the question? For the latter, I recommend BeautifulSoup -- download it and install it, then, once having made a soup object out of the HTML, you can easily locate the tag with a certain id (or other attribute), e.g.:

errp = soup.find(attrs={'id': 'ErrorPanel'})
if errp is not None:
  print 'Error:', errp.string

and similarly for the other case (easily tweakable e.g. into a loop if you're looking for non-unique attributes, and so on).

Answer 2

You can also do it with lxml. It handles HTML very well, and you can use CSS selectors for querying DOM, which makes it particularly attractive if you use libraries like jQuery regularly.