views:

233

answers:

2

I just wanted to develop a translation app in a Django projects which enables registered users with certain permissions to translate every single message it appears in latest version.

My question is, what character set should I use for database tables in this translation app? Looks like some european language characters cannot be stored in UTF-8?

+2  A: 

UTF-8 can be used to represent all of Unicode, so it doesn't let you express all common languages. It allows you to express all languages.

If it seems as if some european characters aren't working, that's an encoding issue.

Williham Totland
+13  A: 

Looks like some european language characters cannot be stored in UTF-8?

Not true. UTF-8 can store any character set without limitations except maybe for Klingon. UTF-8 is your one stop shop for internationalization. If you have problems with characters, they are most likely to be encoding problems, or missing support for that character range in the font you're using to display the data with (Extremely unlikely for a european language character though, but common e.g. when viewing indian sites on an european computer. See also this question)

If a non-western character set can't be rendered, it could be that the user's built in font does not have that range of UTF-8 covered.

Update: Klingon it is indeed not part of official UTF-8:

Some modern invented scripts which have not yet been included in Unicode (e.g., Tengwar) or which do not qualify for inclusion in Unicode due to lack of real-world use (e.g., Klingon) are listed in the ConScript Unicode Registry, along with unofficial but widely-used Private Use Area code assignments.

However, there is a volunteer project that has inofficially assigned code points F8D0-F8FF in the private area to Klingon. Gallery of Klingon characters

Pekka
If someone wrote an addendum to Unicode containing code points for the Klingon alphabet, this could also be expressed with UTF-8. Or you could just use a private use area and make a font that maps to those code points.
Williham Totland
+1 for Klingon :)
Felix Kling
@Williham it's been done: Klingon (F8D0-F8FF) http://en.wikipedia.org/wiki/ConScript_Unicode_Registry
Pekka
+1 for Klingon aswell :D
Acron
What the heck. +1 for Klingon. ;I
Williham Totland