Rendering XML document with multiple languages

Question

I have an XML page with some elements in various languages - Arabic, English, Chinese, Japanese.. Which encoding format should I have to choose for that? If I try to render the XML with an XSL (using utf-8 or ISO-8859-6 or ISO-2022-JP), I get this error:

An invalid character was found in text content.

How shall or solve this?

Thanks.

Answer 1

Where exactly is the error found? It sounds like the XML itself may have an invalid character (e.g. a control character between U+0000 and U+001F other than \r, \t and \n IIRC). You'd probably see this when loading the XML into any decent XML editor (or programmatically).

Aside from that, UTF-8 is generally a good choice of encoding - it's less efficient than UTF-16 for Far East characters, mind you. Both UTF-16 and UTF-8 allow all Unicode characters to be represented (using surrogate pairs in UTF-16 for characters outside the basic multilingual plane).

Answer 2

UTF-8 covers all of the UCS2 (Which is what most people are referring to when they say Unicode) characters, and as such should be appropriate. You still have to make sure there isn't any embedded characters that shouldn't appear in XML such as < or > or non-printable characters

Answer 3

UTF-8 is the only encoding that can handle all those alphabets. It's also the default encoding for XML, and the only encoding that makes sense for a modern application. (For storage/on-the-wire, anyway; for internal processing your language's string type would be more likely to be UTF-16 or 32.)

It would seem from the error that you have a problem in the input file, rather than an issue with your choice of output encoding. Maybe it's encoded in something other than UTF-8 but has forgotten to include an <?xml encoding?> declaration to say so. Or maybe there's an invalid ISO-2202-JP escape sequence? (This is a horror of an encoding.)

You should try to load the input file into something that parses XML (eg. Firefox or IE) and see what errors, if any, it comes up with.

(You can't mix encodings in a single XML file. If you've spat out bytes strings from different sources into XML, you've already lost. How is this XML generated?)