Hi
I have downloaded page using urlopen. How do I remove all html tags from it? Is there any regexp to replace all <*> tags?
A:
A very simple regexp would be :
import re
notag = re.sub("<.*?>", " ", html)
The drawback of this solution is that it doesn't remove javascript or css, but only tags.
Guillaume Lebourgeois
2010-07-28 09:24:09
That one will fail on you if you have non-escaped '<' and '>' characters outside or inside of the actual tags. If that is not an issue, you can use a regexp. If not, you will need to use an actual html/xml parser.
relet
2010-07-28 09:25:13
-1: Ill-advised, brittle solution. It will be hard to debug the page with one extra "<" or ">" that breaks this.
S.Lott
2010-07-28 11:02:08
It's simple and efficient, a great of cleaning data simply, I personnaly use it daily. When it comes to a more complicated problems, I then use BeautifulSoup, but please : let's keep simple problems _simples_. The case you're evoking here will be rare, if not unexistant. Taking it into account would be necessary ONLY if you need ALL your potential data, and you need it absolutely clean.
Guillaume Lebourgeois
2010-07-28 12:21:15
There are only about a thousand ways this can fail, and absolutely will for real web pages.
bobince
2010-07-28 12:21:33
@bobince I personnaly don't work with fake web pages.
Guillaume Lebourgeois
2010-07-28 12:35:34
A:
Try this:
import re
def remove_html_tags(data):
p = re.compile(r'<.*?>')
return p.sub('', data)
Martin
2010-07-28 09:24:48
+9
A:
I can also recommend BeautifulSoup which is an easy to use html parser. There you would do something like:
from BeautifulSoup import BeautifulSoup
soup = BeautifulSoup(html)
all_text = ''.join(soup.findAll(text=True))
This way you get all the text from a html document.
Uli Held
2010-07-28 09:50:56
This requires an external library which is not always a solution to meet the needs of certain folk where they wish to distribute the script.
Martin
2010-08-04 15:45:25