parsing XML with ampersand

Question

I have a string which contains XML, I just want to parse into Xelement, but it has an ampersand. I still have problem to parse it with HtmlDecode. Any suggestion?

string test = " <MyXML><SubXML><XmlEntry Element="test" value="wow&" /></SubXML></MyXML>"; 

XElement.Parse(HttpUtility.HtmlDecode(test));

I also added these methods to replace those characters, but I am still getting XMLException.

string encodedXml = test.Replace("&", "&amp;").Replace("<", "&lt;").Replace(">", "&gt;").Replace("\"", "&quot;").Replace("'", "&apos;");
XElement myXML = XElement.Parse(encodedXml);

t or Even tried it with this:

string newContent=  SecurityElement.Escape(test);
XElement myXML = XElement.Parse(newContent);

Answer 1

If your string is not valid XML, it will not parse. If it contains an ampersand on its own, it's not valid XML. Contrary to HTML, XML is very strict.

Answer 2

The ampersant makes the XML invalid. This cannot be fixed by a stylesheet so you need to write code with some other tool or code in VB/C#/PHP/Delphi/Lisp/Etc. to remove it or to translate it to &.

Answer 3

Your string doesn't contain valid XML, that's the issue. You need to change your string to:

<MyXML><SubXML><XmlEntry Element="test" value="wow&amp;" /></SubXML></MyXML>"

Answer 4

You should 'encode' rather than decode. But calling HttpUtility.HtmlEncode will not help you as it will encode your '<' and '>' symbols as well and your string will no longer be an XML.

I think that for this case the best solution would be to replace '&' with '& amp;' (with no space)

Answer 5

HtmlEncode will not do the trick, it will probably create even more ampersands (for instance, a ' might become ", which is an Xml entity reference, which are the following:

&   & 
'  ' 
"  " 
&lt;    < 
&gt;    >

But it might you get things like &nbsp, which is fine in html, but not in Xml. Therefore, like everybody else said, correct the xml first by making sure any character that is NOT PART OF THE ACTUAL MARKUP OF YOUR XML (that is to say, anything INSIDE your xml as a variable or text) and that occurs in the entity reference list is translated to their corresponding entity (so < would become <). If the text containing the illegal character is text inside an xml node, you could take the easy way and surround the text with a CDATA element, this won't work for attributes though.

Answer 6

Ideally the XML is escaped properly prior to your code consuming it. If this is beyond your control you could write a regex. Do not use the String.Replace method unless you're absolutely sure the values do not contain other escaped items.

For example, "wow&".Replace("&", "&") results in wow& which is clearly undesirable.

Regex.Replace can give you more control to avoid this scenario, and can be written to only match "&" symbols that are not part of other characters, such as <, something like:

string result = Regex.Replace(test, "&(?!(amp|apos|quot|lt|gt);)", "&");

The above works, but admittedly it doesn't cover the variety of other characters that start with an ampersand, such as   and the list can grow.

A more flexible approach would be to decode the content of the value attribute, then re-encode it. If you have value="&wow&" the decode process would return "&wow&" then re-encoding it would return "&wow&", which is desirable. To pull this off you could use this:

string result = Regex.Replace(test, @"value=\""(.*?)\""", m => "value=\"" +
    HttpUtility.HtmlEncode(HttpUtility.HtmlDecode(m.Groups[1].Value)) +
    "\"");
var doc = XElement.Parse(result);

Bear in mind that the above regex only targets the contents of the value attribute. If there are other areas in the XML structure that suffer from the same issue then it can be tweaked to match them and replace their content in a similar fashion.

---

EDIT: updated solution that should handle content between tags as well as anything between double quotes. Be sure to test this thoroughly. Attempting to manipulate XML/HTML tags with regex is not favorable as it can be error prone and over-complicated. Your case is somewhat special since you need to sanitize it first in order to make use of it.

string pattern = "(?<start>>)(?<content>.+?(?<!>))(?<end><)|(?<start>\")(?<content>.+?)(?<end>\")";
string result = Regex.Replace(test, pattern, m =>
      m.Groups["start"].Value +
      HttpUtility.HtmlEncode(HttpUtility.HtmlDecode(m.Groups["content"].Value)) +
      m.Groups["end"].Value);
var doc = XElement.Parse(result);

Answer 7

Perhaps consider writing your own XMLDocumentScanner. That's what NekoHTML is doing to have the ability to ignore ampersands not used as entity references.