What is a light python library that can eliminate HTML tags? (and only text)

Question

I know that NLTK has it. But any else?

Answer 1

python standard module html.parser should allow you to parse simple html content and eliminate tags. you only have to derive HTMLParser, then overload all handle_*() methods so that they output or discard content, depending on the surrounding element tags.

Answer 2

BeautifulSoup: http://www.crummy.com/software/BeautifulSoup/

From the home page:

Beautiful Soup is a Python HTML/XML parser designed for quick turnaround projects like screen-scraping. Three features make it powerful:

1. Beautiful Soup won't choke if you give it bad markup. It yields a parse tree that makes approximately as much sense as your original document. This is usually good enough to collect the data you need and run away.
2. Beautiful Soup provides a few simple methods and Pythonic idioms for navigating, searching, and modifying a parse tree: a toolkit for dissecting a document and extracting what you need. You don't have to create a custom parser for each application.
3. Beautiful Soup automatically converts incoming documents to Unicode and outgoing documents to UTF-8. You don't have to think about encodings, unless the document doesn't specify an encoding and Beautiful Soup can't autodetect one. Then you just have to specify the original encoding.

Answer 3

If your licensing permits it, you could use html2text (the asciinator) (GPL).

Answer 4

You may want to a look at Strip-o-Gram HTML Conversion Library: http://pypi.python.org/pypi/stripogram/1.5

example usage from readme.txt:

  from stripogram import html2text, html2safehtml
  mylumpofdodgyhtml # a lump of dodgy html ;-)
  # Only allow <b>, <a>, <i>, <br>, and <p> tags
  mylumpofcoolcleancollectedhtml = html2safehtml(mylumpofdodgyhtml,valid_tags=("b", "a", "i", "br", "p"))
  # Don't process <img> tags, just strip them out. Use an indent of 4 spaces 
  # and a page that's 80 characters wide.
  mylumpoftext = html2text(mylumpofcoolcleancollectedhtml,ignore_tags=("img",),indent_width=4,page_width=80)