UTF-8 output on Windows XP console

Question

The following code shows unexpected behaviour on my machine (I'm using Visual C++ 2008 SP1 on Windows XP here):

int main() {
      SetConsoleOutputCP( CP_UTF8 );
      std::cout << "\xc3\xbc";
      int fail = std::cout.fail() ? '1': '0';
      fputc( fail, stdout );
      fputs( "\xc3\xbc", stdout );
}

I simply compiled with cl /EHsc test.cpp. Output in a console window is ü0ü (translated to Codepage 1252, originally shows some line drawing charachters in the default Codepage, perhaps 437). When I change the settings of the console window to use the "Lucida Console" character set and run my test.exe again, output is changed to 1ü, which means

• the character ü can be written using fputs and its UTF-8 encoding C3 BC
• std::cout does not work for whatever reason
• the streams failbit is setting after trying to write the character

I tried to raise this issue on "Microsoft Connect" (see here), but MS has not been very helpful. You might as well look here as something similar has been asked before.

Can you reproduce this problem?

What am I doing wrong? Shouldn't the std::cout and the fputs have the same effect?

Answer 1

Oi. Congratulations on finding a way to change the code page of the console from inside your program. I didn't know about that call, I always had to use chcp.

I'm guessing the C++ default locale is getting involved. By default it will use the code page provide by GetThreadLocale() to determine the text encoding of non-wstring stuff. This generally defaults to CP1252. You could try using SetThreadLocale() to get to UTF-8 (if it even does that, can't recall), with the hope that std::locale defaults to something that can handle your UTF-8 encoding.