Unicode RTF text in RichEdit

Question

I'm having trouble getting a RichEdit control to display unicode RTF text. My application is Unicode, so all strings are wchar_t strings.
If I create the control as "RichEdit20A" I can use e.g. SetWindowText, and the text is displayed with the proper formatting. If I create the control as "RichEdit20W" then using SetWindowText shows the text verbatim, i.e. all the RTF code is displayed. The same happens if I use the EM_SETTEXTEX parameter, specifying codepage 1200 which MSDN tells me is used to indicate unicode.
I've tried using the StreamIn function, but this only seems to work if I stream in ASCII text. If I stream in widechars then I get empty text in the control. I use the SF_RTF|SF_UNICODE flags, and MSDN hints that this combination may not be allowed.

So what to do? Is there any way to get widechars into a RichEdit without losing RTF interpretation, or do I need to encode it? I've thought about trying UTF-8, or perhaps use the encoding facilities in RTF, but am unsure what the best choice is.

Answer 1

I had to do this recently, and noticed the same sorts of observations you're making.

It seems that, despite what MSDN almost suggests, the "RTF" parser will only work with 8-bit encodings. So what I ended up doing was using UTF-8, which is an 8 bit encoding but still can represent the full range of Unicode characters. You can get UTF-8 from a PWSTR via WideCharToMultiByte():

PWSTR WideString = /* Some string... */;
DWORD WideLength = wcslen(WideString) + 1;
PSTR Utf8;
DWORD Length;
INT ReturnedLength;

// A utf8 representation shouldn't be longer than 4 times the size
// of the utf16 one.
Length = WideLength * 4;
Utf8 = malloc(Length);
if (!Utf8) { /* TODO: handle failure */ }

ReturnedLength = WideCharToMultiByte(CP_UTF8,
                                     0,
                                     WideString,
                                     WideLength-1,
                                     Utf8,
                                     Length-1,
                                     NULL,
                                     NULL);
if (ReturnedLength)
{
   // Need to zero terminate...
   Utf8[ReturnedLength] = 0;
}
else { /* TODO: handle failure */ }

Once you have it in UTF-8, you can do:

SETTEXTEX TextInfo = {0};

TextInfo.flags = ST_SELECTION;
TextInfo.codepage = CP_UTF8;

SendMessage(hRichText, EM_SETTEXTEX, (WPARAM)&TextInfo, (LPARAM)Utf8);

And of course (I left this out originally, but while I'm being explicit...):

free(Utf8);

Answer 2

RTF is ASCII, any charactor out of ASCII would be encoded using escape sequence. RTF 1.9.1 specification (March 2008)