Ignoring XML errors in Python

Question

I am using XML minidom (xml.dom.minidom) in Python, but any error in the XML will kill the parser. Is it possible to ignore them, like a browser for example? I am trying to write a browser in Python, but it just throws an exception if the tags aren't fully compatible.

Answer 1

There is a library called BeautifulSoup, I think it's what you're looking for. As you're trying to parse a invalid XML, the normal XML parser won't work. BeautifulSoup is more fail-tolerant, it can still extract information from invalid XML.

Beautiful Soup is a Python HTML/XML parser designed for quick turnaround projects like screen-scraping. Three features make it powerful:

1. Beautiful Soup won't choke if you give it bad markup. It yields a parse tree that makes approximately as much sense as your original document. This is usually good enough to collect the data you need and run away.
2. Beautiful Soup provides a few simple methods and Pythonic idioms for navigating, searching, and modifying a parse tree: a toolkit for dissecting a document and extracting what you need. You don't have to create a custom parser for each application.
3. Beautiful Soup automatically converts incoming documents to Unicode and outgoing documents to UTF-8. You don't have to think about encodings, unless the document doesn't specify an encoding and Beautiful Soup can't autodetect one. Then you just have to specify the original encoding.

Beautiful Soup parses anything you give it, and does the tree traversal stuff for you. You can tell it "Find all the links", or "Find all the links of class externalLink", or "Find all the links whose urls match "foo.com", or "Find the table heading that's got bold text, then give me that text."

Answer 2

See, for example, extracting-text-from-html-file-using-python for suggestions regarding ways for parsing HTML in Python.

Answer 3

It should be noted that while HTML looks like XML it is not XML. XHTML is an XML form of HTML.