ansaurus

Question

Parse html and find data in the html

Answer 1

A:

try using jquery. and you can retrieve all elements. alternately, you can put an id on your row and pull it out.

1) ... ...

$("td")[1].innerHTML will be what you want

2) ... ...

$("#blah").text() will be what you want

yamspog 2010-04-01 04:30:37

I think the request was for a Python solution.

Greg 2010-04-01 04:47:17

Answer 2

A:

i believe you can do css search on lxml objects.. like so

elements = root.cssselect('div.content')
data = elements[0].text

z33m 2010-04-01 04:33:36

Answer 3

+1 A:

With BeautifulSoup, you can do that with

>>> soup = BeautifulSoup.BeautifulSoup('<html><table><tr><td>Header</td></tr><tr><td>Want This</td></tr></table></html>')
>>> soup.findAll('td')[1].string
u'Want This'
>>> soup.findAll('tr')[1].td.string
u'Want This'

(Obviously that's a really crude example, but ya.)

isbadawi 2010-04-01 04:36:17

Answer 4

A:

You can use xml.dom.minidom:

from xml.dom.minidom import parseString

doc = parseString('<html><table><tr><td>Header</td></tr><tr><td>Want This</td> </tr></table></html>')

def parse(node):
    for node in node.childNodes:
        print(node)
        parse(node)

parse(doc)

There are a number of ways to access the 1st column of the 2nd row. You can iterate through the document until you come across it, you can jump straight to it via attributes, etc... The module is documented very well.

http://docs.python.org/py3k/library/xml.dom.html

http://docs.python.org/py3k/library/xml.dom.minidom.html

rcoyner 2010-04-01 04:41:45

but what happens of the html is not correct xml?

Mark 2010-04-05 11:34:36

Answer 5

+5 A:

Lack of documentation is a good reason to avoid a library IMO, no matter how cool it is. Are you wedded to using html5lib? Have you looked at lxml.html?

Here is a way to do this with lxml:

from lxml import html
tree = html.fromstring(text)
[td.text for td in tree.xpath("//td")]

Result:

['Header', 'Want This']

Ryan Ginstrom 2010-04-01 05:13:30

Answer 6

+1 A:

I always recommend to try out lxml library. It's blazingly fast and has many features.

It has also support for html5lib parser if you need that: html5parser

>>> from lxml.html import fromstring, tostring

>>> html = """
... <html>
...     <table>
...         <tr><td>Header</td></tr>
...         <tr><td>Want This</td></tr>
...     </table>
... </html>
... """
>>> doc = fromstring(html)
>>> tr = doc.cssselect('table tr')[1]
>>> print tostring(tr)
<tr><td>Want This</td></tr>

Ruslan Spivak 2010-04-01 05:17:53

This is how I'd do it, except I'd use "print doc.cssselect('tr')[1].text_content()" to get at the contents of the second row, rather than have lxml show the HTML.

Greg 2010-04-01 14:34:02

ansaurus

tags:

views:

answers:

Parse html and find data in the html

related questions