javascript json - problem decoding ajax json array from php

Question

Hello,

I'm using php's json_encode() to convert an array to json which then echo's it and is read from a javascript ajax request.

The problem is the echo'd text has unicode characters which the javascript json parse() function doesn't convert to.

Example array value is "2\u00000\u00001\u00000\u0000-\u00001\u00000\u0000-\u00000\u00001" which is "2010-10-01".

Json.parse() only gives me "2".

Anyone help me with this issue?

Example:

            var resArray = JSON.parse(this.responseText);
            for(var x=0; x < resArray.length; x++) {
                var twt = resArray[x];
                alert(twt.date);
                break;
            }

Answer 1

You have NUL characters (character code zero) in the string. It's actually "2_0_1_0_-_1_0_-_0_1", where _ represents the NUL characters.

The unicode character escape is actually part of the JSON standard, so the parser should handle that correctly. However, the result is still a string will NUL characters in it, so when you try to use the string in Javascript the behaviour will depend on what the browser does with the NUL characters.

You can try this in some different browsers:

alert('as\u0000df');

Internet Explorer will display only as

Firefox will display asdf but the NUL character doesn't display.

The best solution would be to remove the NUL characters before you convert the data to JSON.

Answer 2

To add to what Guffa said:

When you have alternating zero bytes, what has almost certainly happened is that you've read a UTF-16 data source without converting it to an ASCII-compatible encoding such as UTF-8. Whilst you can throw away the nulls, this will mangle the string if it contains any characters outside of ASCII range. (Not an issue for date strings of course, but it may affect any other strings you're reading from the same source.)

Check where your PHP code is reading the 2010-10-01 string from, and either convert it on the fly using eg iconv('utf-16le', 'utf-8', $string), or change the source to use a more reasonable encoding. If it's a text file, for example, save it in a text editor using ‘UTF-8 without BOM’, and not ‘Unicode’, which is a highly misleading name Windows text editors use to mean UTF-16LE.