unicode in powershell with python? alternative shells in windows?

Question

I want a shell that supports unicode in windows, powershell as it ships doesn't seem to. Powershell V2 (win7 x64) :

PS C:\> powershell
Windows PowerShell
Copyright (C) 2009 Microsoft Corporation. All rights reserved.

PS C:\> python
Python 2.6.2 (r262:71605, Apr 14 2009, 22:46:50) [MSC v.1500 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> unicode_char=unichr(0xf12)
>>> unicode_char
u'\u0f12'
>>> print unicode_char
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\python26\lib\encodings\cp437.py", line 12, in encode
    return codecs.charmap_encode(input,errors,encoding_map)
UnicodeEncodeError: 'charmap' codec can't encode character u'\u0f12' in position 0: character maps to <undefined>
>>>

I get similar results with PowerShell ISE, even though some places around the web claim it to be unicode supporting or whatever...

the Python Integrated Developement Environment (IDLE) 2.6.2 seems to work fine:

>>> unicode_char=unichr(0xf12)
>>> print unicode_char
༒
>>> 

IDLE is very slow and would prefer another shell, any ideas? Can I make this work in powershell?

Answer 1

Maybe you should try ipython?

Answer 2

The Windows console subsystem is not Unicode but code page based. You can play around with setting the code page:

PS> chcp 65001
PS> ipy64.exe
>>> print unichr(0x3a9)
Ω

I couldn't get (0xf12) to give the right character with that codepage. Perhaps it is available on another code page.

ISE can display unicode and accept unicode input e.g.:

PS> [char]0xf12
༒
PS> [char]0xe4
ä
PS> [char]0x3a9
Ω

However ISE doesn't seem to play well with the IronPython interpreter.

Further to the point that ISE seems to handle Unicode coming from a native app via stdout:

$src = @'
namespace Foo {
    public class Bar
    {
        public static void Baz()
        {
            System.Console.Out.WriteLine("\u0f12");
            System.Console.Out.WriteLine("\u00e4");
            System.Console.Out.WriteLine("\u03a9");
        }
    }
}
'@

Add-Type -TypeDefinition $src

[Foo.Bar]::Baz()
༒
ä
Ω

Answer 3

The PowerShell itself is Unicode, but the clasic console has problems with Unicode. But PowerShell ISE is definitely Unicode.

Try this:

PS C:\> $a = [char]0xf12
PS C:\> echo $a

This is Python not playing nice.

You can try chcp 65001 before starting python (to set the code page to UTF-8). No promise though (I did not try to see if it works, no python installed on this machine)

Answer 4

Note that, when issuing chcp 65001 in a Windows console the font must be Lucida Console and not any of the bitmap fonts. Theoretically, other monospaced ttf fonts should work, but they don't (practically). It's an issue of attributes that MS checks in monospaced fonts intended for the console, and Lucida Console includes them.

There is at least one issue open for the Python Windows console unicode problems here.