+7  A: 

This has nothing to do with Perl's internal string encoding, but with the need to properly decode data coming from the outside. I'll provide the test case. This is Strawberry Perl 5.10 on a Western European Windows XP.

test1.pl:

use Devel::Peek;
print Dump $ENV{TEST};
use Encode qw(decode);
my $var = decode 'Windows-1252', $ENV{TEST};
print Dump $var;

system "B:/sperl/perl/bin/perl.exe B:/test2.pl";

test2.pl:

use Devel::Peek;
print Dump $ENV{TEST};
use Encode qw(decode);
my $var = decode 'IBM850', $ENV{TEST};
# using Windows-1252 again is wrong here
print Dump $var;

Execute:

> set TEST=abc£
> B:\sperl\perl\bin\perl.exe B:\test1.pl

Output (shortened):

SV = PVMG(0x982314) at 0x989a24
  FLAGS = (SMG, RMG, POK, pPOK)
  PV = 0x98de0c "abc\243"\0
SV = PV(0x3d6a64) at 0x989b04
  FLAGS = (PADMY, POK, pPOK, UTF8)
  PV = 0x9b5be4 "abc\302\243"\0 [UTF8 "abc\x{a3}"]
SV = PVMG(0x982314) at 0x989a24
  FLAGS = (SMG, RMG, POK, pPOK)
  PV = 0x98de0c "abc\243"\0
SV = PV(0x3d6a4c) at 0x989b04
  FLAGS = (PADMY, POK, pPOK, UTF8)
  PV = 0x9b587c "abc\302\243"\0 [UTF8 "abc\x{a3}"]

You are bitten by the fact that Windows uses a different encoding for the text environment (IBM850) than for the graphical environment (Windows-1252). An expert has to explain the deeper details of that phenomenon.

Edit:

It is possible to heuristically (meaning it will fail to do the right thing sometimes, especially for such short strings) determine encodings. The best general purpose solution is Encode::Detect/Encode::Detect::Detector which is based on Mozilla nsUniversalDetector.

There are some ways to decode external data implicitely such as the open pragma/IO layers and the -C switch, however they deal with file streams and program arguments only. As of now, from the environment must be decoded explicitely. I like that better anyway, explicite shows the maintainance programmer you have thought the topic through.

daxim
Thanks daxim for your reply.Few questions -1) Is is possible to determine what encoding is used for string that come from outside world? This would help to decode accordingly in perl.2) Is there a way to instruct perl to handle the decoding part internally rather than we doing by ourself determining the encoding used in external world?-Kartlee
Kartlee
Thanks for the reminder. I did now for what I think is acceptable.
Kartlee