Hello, i have need webpage-content. I need to get some data from it. It looks like:
< div class="deg">DATA< /div>
As i understand, i have to use regex, but i can't choose one.
I tried the code below but had no any results. Please, correct me:
regexHandler = re.compile('(<div class="deg">(?P<div class="deg">.*?)</div>)')
result = regexHandler.search( pageData )
If you want the div tags included in the matched item:
regexpHandler = re.compile('(<div class="deg">.*?</div>)')
If you don't want the div tags included, only the DATA portion:
regexpHandler = re.compile('<div class="deg">(.*?)</div>')
Then to run the match and get the result:
result = regexHandler.search( pageData )
matchedText = result.groups()[0]
I suggest using a good HTML parser (such as BeautifulSoup -- but for your purposes, i.e. with well-formed HTML as input, the ones that come with the Python standard library, such as HTMLParser, should also work well) rather than raw REs to parse HTML.
If you want to persist with the raw RE approach, the pattern:
r'<div class="deg">([^<]*)</div>'
looks like the simplest way to get the string 'DATA' out of the string '<div class="deg">DATA</div>' -- assuming that's what you're after. You may need to add one or more \s* in spots where you need to tolerate optional whitespace.
you can use simple string functions in Python, no need for regex
mystr = """< div class="deg">DATA< /div>"""
if "div" in mystr and "class" in mystr and "deg" in mystr:
s = mystr.split(">")
for n,item in enumerate(s):
if "deg" in item:
print s[n+1][:s[n+1].index("<")]
my approach, get something to split on. eg in the above, i split on ">". Then go through the splitted items, check for "deg", and get the item after it, since "deg" appears before the data you want to get. of course, this is not the only approach.
While it is ok to use rexex for quick and dirty html processing a much better and cleaner way is to use a html parser like lxml.html and to query the parsed tree with XPath or CSS Selectors.
html = """<html><body><div class="deg">DATA1</div><div class="deg">DATA2</div></body></html>"""
import lxml.html
page = lxml.html.fromstring(html)
#page = lxml.html.parse(url)
for element in page.findall('.//div[@class="deg"]'):
print element.text
#using css selectors
from lxml.cssselect import CSSSelector
sel = CSSSelector("div.deg")
for element in sel(page):
print element.text