Python regex: how to extract inner data from regex

Question

Hi! I want to extract data from such regex:

<td>[a-zA-Z]+</td><td>[\d]+.[\d]+</td><td>[\d]+</td><td>[\d]+.[\d]+</td>  

I've found related question http://stackoverflow.com/questions/1327369/python-extract-contents-of-regex but in my case I shoud iterate somehow.

Answer 1

As paprika mentioned in his/her comment, you need to identify the desired parts of any matched text using ()'s to set off the capture groups. To get the contents from within the td tags, change:

<td>[a-zA-Z]+</td><td>[\d]+.[\d]+</td><td>[\d]+</td><td>[\d]+.[\d]+</td> 

to:

<td>([a-zA-Z]+)</td><td>([\d]+.[\d]+)</td><td>([\d]+)</td><td>([\d]+.[\d]+)</td>
     ^^^^^^^^^           ^^^^^^^^^^^           ^^^^^           ^^^^^^^^^^^
      group 1             group 2              group 3          group 4

And then access the groups by number. (Just the first line, the line with the '^'s and the one naming the groups are just there to help you see the capture groups as specified by the parentheses.)

dataPattern = re.compile(r"<td>[a-zA-Z]+</td>... etc.")
match = dataPattern.find(htmlstring)
field1 = match.group(1)
field2 = match.group(2)

and so on. But you should know that using re's to crack HTML source is one of the paths toward madness. There are many potential surprises that will lurk in your input HTML, that are perfectly working HTML, but will easily defeat your re:

• "<TD>" instead of "<td>"

• spaces between tags, or between data and tags

• "&nbsp;" spacing characters

Libraries like BeautifulSoup, lxml, or even pyparsing will make for more robust web scrapers.

Answer 2

As the poster clarified, the <td> tags should be removed from the string.

Note that the string you've shown us is just that: a string. Only if used in the context of regular expression functions is it a regular expression (a regexp object can be compiled from it).

You could remove the <td> tags as simply as this (assuming your string is stored in s): s.replace('<td>','').replace('</td>','')

Watch out for the gotchas however: this is really of limited use in the context of real HTML, just as others pointed out.

Further, you should be aware that whatever regular expression [string] is left, what you can parse with that is probably not what you want, i.e. it's not going to automatically match anything that it matched before without <td> tags!