How do I convert unicode codepoints to hexadecimal HTML entities?

Question

I have a data file (an Apple plist, to be exact), that has Unicode codepoints like \U00e8 and \U2019. I need to turn these into valid hexadecimal HTML entities using PHP.

What I'm doing right now is a long string of:

 $fileContents = str_replace("\U00e8", "è", $fileContents);
 $fileContents = str_replace("\U2019", "’", $fileContents);

Which is clearly dreadful. I could use a regular expression to convert the \U and all trailing 0s to &#x, then stick on the trailing ;, but that also seems heavy-handed.

Is there a clean, simple way to take a string, and replace all the unicode codepoints to HTML entities?

Answer 1

You can use preg_replace:

preg_replace('/\\\\U0*([0-9a-fA-F]{1,5})/', '&#x\1;', $fileContents);

Testing the RE:

PS> 'some \U00e8 string with \U2019 embedded Unicode' -replace '\\U0*([0-9a-f]{1,5})','&#x$1;'
some è string with ’ embedded Unicode

Answer 2

Here's a correct answer, that deals with the fact that those are code units, not code points, and allows unencoding supplementary characters.

function unenc_utf16_code_units($string) {
    /* go for possible surrogate pairs first */
    $string = preg_replace_callback(
        '/\\\\U(D[89ab][0-9a-f]{2})\\\\U(D[c-f][0-9a-f]{2})/i',
        function ($matches) {
            $hi_surr = hexdec($matches[1]);
            $lo_surr = hexdec($matches[2]);
            $scalar = (0x10000 + (($hi_surr & 0x3FF) << 10) |
                ($lo_surr & 0x3FF));
            return "&#x" . dechex($scalar) . ";";
        }, $string);
    /* now the rest */
    $string = preg_replace_callback('/\\\\U([0-9a-f]{4})/i',
        function ($matches) {
            //just to remove leading zeros
            return "&#x" . dechex(hexdec($matches[1])) . ";";
        }, $string);
    return $string;
}