How can I create a Unicode character from its bytes when they are stored in different variables in Perl?

Question

I am trying to Convert hex representations of Unicode characters to the characters they represent. The following example works fine:

#!/usr/bin/perl

use Encode qw( encode decode );
binmode(STDOUT, ':encoding(utf-8)');

my $encoded = encode('utf8', "\x{e382}\x{af}");
eval { $encoded = decode('utf8', $encoded, Encode::FB_CROAK); 1 }
or print("coaked\n");

print "$encoded\n";

However the hex digits are stored in 3 variables.

So if i replace the encode line with this:

my $encoded = encode('utf8', "\x{${byte1}${byte2}}\x{${byte3}}");

where

my $byte1 = "e3"; my $byte2 = "82"; my $byte3 = "af";

It fails as it tries to evaluate the \x immediately and sees the $ sign and { as characters.

Does anyone know how to get around this.

Answer 1

Instead of

my $encoded = encode('utf8', "\x{${byte1}${byte2}}\x{${byte3}}");

You can use

my $encoded = encode('utf8', chr(hex($byte1 . $byte2)) . chr(hex($byte3)));

hex() converts from hexadecimal, and chr() returns the unicode character for a given code point.

[Edit:]

Not related to your question, but I noticed you mix utf-8 and utf8 in your program. I don't know if this is a typo, but you should be a ware that these are not the same things in Perl:
utf-8 (with hyphen, case insensitive) is what the UTF-8 standard says, whereas utf8 (no hyphen, also case insensitive) is Perls internal encoding, which is more loosely defined (it allows codepoints that are not valid unicode codepoints). In general, you should stick to utf-8 (perlunifaq has the details).

Answer 2

trendel's answer seems pretty good, but Encode::Escape offers an alternative solution:

use Encode::Escape::Unicode;

my $hex = '263a';
my $escaped = "\\x{" . $hex . "}\n";
print encode 'utf8', decode 'unicode-escape', $escaped;

Answer 3

First off, think hard about why you ended up with three variables, $byte1, $byte2, $byte3, each holding one byte's worth of data, as a two-character string, in hex. This part of your program seems hard because of a poor design decision further up. Fix that bad decision, and this part of the code will fall out naturally.

That being said, what you want to do, I think, is this:

my $byte1 = "e3"; my $byte2 = "82"; my $byte3 = "af";
my $str = chr(hex($byte1 . $byte2)) . chr(hex($byte3))

The encoding stuff is a red herring; you shouldn't be worrying about encodings in the middle of your program, only when you do IO.

I'm assuming in the above that you want to get out a two character string, U+E382 followed by U+AF. That's what you actually asked for. However, since there is no U+E382, since it's in the middle of the private use area, that's probably not what you actually wanted. Please try to reword the question? Perhaps ask a more basic question, and describe what you are trying to achieve, rather then how you are going about trying to do it?