how to convert utf-8 to ASCII in c++?

Question

i am getting response from server in utf-8 but not able to read that. how to convert utf-8 to ASCII in c++?

Answer 1

UTF-8 is backwards compatible with ASCII meaning all ASCII characters are encoded as single unchanged byte values in UTF-8. If the text should be ASCII but you are unable to read it then there must be another issue.

Answer 2

UTF-8 is an encoding that can map every unicode character. ASCII only supports a very small subset of unicode.

For the subset of unicode that is ASCII, the mapping from UTF-8 to ASCII is a direct one-to-one byte mapping, so if the server sends you a document that only contains ASCII characters in UTF-8 encoding then you can directly read that as ASCII.

If the response contains non-ASCII characters then, whatever you do, you won't be able to express them in ASCII. To filter these out of a UTF-8 stream you can just filter out any byte >= 128 (0x80 hex).

Answer 3

First note that ASCII is a 7-bit format. There are 8-bit encodings, if you are after one of these (such as ISO 8859-1) you'll need to be more specific.

To convert an ASCII string to UTF-8, do nothing: they are the same. So if your UTF-8 string is composed only of ASCII characters, then it is already an ASCII string, and no conversion is necessary.

If the UTF-8 string contains non-ASCII characters (anything with accents or non-Latin characters), there is no way to convert it to ASCII. (You may be able to convert it to one of the ISO encodings perhaps.)

There are ways to strip the accents from Latin characters to get at least some resemblance in ASCII. Alternatively if you just want to delete the non-ASCII characters, simply delete all bytes with values >= 128 from the utf-8 string.

Answer 4

Here you can find some example code:

http://www.unix.com/programming/10594-howto-convert-ascii-utf-8-back-c.html

Answer 5

ASCII is a codepage representing 128 characters and control codes where as utf8 is able to represent any character in the Unicode standard which is much -much more to ASCII capabilities. So Answer to your Question is : Not Possible Unless you have any more specification for the data source.

Answer 6

Check this utf-8 String Library, forget about converting it to ASCII.

Answer 7

This example works under Windows (you did not mention your target operating system):

    // The sample buffer contains "©ha®a©te®s" in UTF-8
    unsigned char buffer[15] = { 0xc2, 0xa9, 0x68, 0x61, 0xc2, 0xae, 0x61, 0xc2, 0xa9, 0x74, 0x65, 0xc2, 0xae, 0x73, 0x00 };
    // utf8 is the pointer to your UTF-8 string
    char* utf8 = (char*)buffer;
    // convert multibyte UTF-8 to wide string UTF-16
    int length = MultiByteToWideChar(CP_UTF8, 0, (LPCSTR)utf8, -1, NULL, 0);
    if (length > 0)
    {
        wchar_t* wide = new wchar_t[length];
        MultiByteToWideChar(CP_UTF8, 0, (LPCSTR)utf8, -1, wide, length);

        // convert it to ANSI, use setlocale() to set your locale, if not set
        size_t convertedChars = 0;
        char* ansi = new char[length];
        wcstombs_s(&convertedChars, ansi, length, wide, _TRUNCATE);
    }

Remember to delete[] wide; and/or ansi when no longer needed. Since this is unicode, I'd recommend to stick to wchar_t* instead of char* unless you are certain that input buffer contains characters that belong to the same ANSI subset.