How do I write a std::codecvt facet?

Question

How do I write a std::codecvt facet? I'd like to write ones that go from UTF-16 to UTF-8, which go from UTF-16 to the systems current code page (windows, so CP_ACP), and to the system's OEM codepage (windows, so CP_OEM).

Cross-platform is preferred, but MSVC on Windows is fine too. Are there any kinds of tutorials or anything of that nature on how to correctly use this class?

Answer 1

The problem with this std::codecvt is it's a solution looking for a problem. Or rather, the problem it's trying to solve is unsolvable, so anybody trying to use it as a solution is going to be very disappointed.

If you don't know which character set your input or output is, then std::codecvt isn't ever going to be able to help you. Conversely, if you do know which character sets you're using, then you can trivially convert between them with a single function call. Wrapping that function call in a complicated mess of templates doesn't change those fundamentals.

...and that's why nobody uses std::codecvt. I recommend you just do what everybody else does, and pretend it never happened.

Answer 2

I've written one based on iconv. It can be used on windows or on any POSIX OS. (You will need to link with iconv obviously).

Enjoy

The answer for the "how to" question is to follow the codecvt reference. I was not able to find any better instructions in the Internet two yars ago.

Important notices

• theoretically there is no need for such work. codecvt_byname should be enough on any standard supporting platform. But in reality there are some compilers that don't support or badly support this class. There is also a difference in interfaces of codecvt_byname on different compilers.
• my working example is implemented with state template parameter of codecvt. Always use standard mbstate type there as this is the only way to use your codecvt with standard iostream classes.
• std::mbstate_t type can't be used as a pointer on 64bit platforms in a cross-platform way.
• stateless conversions work for short strings, but may fail if you try to convert a data chunk greater that streambuf internal buffer size (UTF is essentially stateful encoding)