Display ñ on a C# .NET application

Question

I have a localization issue.

One of my industrious coworkers has replaced all the strings throughout our application with constants that are contained in a dictionary. That dictionary gets various strings placed in it once the user selects a language (English by default, but target languages are German, Spanish, French, Portuguese, Mandarin, and Thai).

For our test of this functionality, we wanted to change a button to include text which has a ñ character, which appears both in Spanish and in the Arial Unicode MS font (which we're using throughout the application).

Problem is, the ñ is appearing as a square block, as if the program did not know how to display it. When I debug into that particular string being read from disk, the debugger reports that character as a square block as well.

So where is the failure? I think it could be in a few places:

1) Notepad may not be unicode aware, so the ñ displayed there is not the same as what vs2008 expects, and so the program interprets the character as a square (EDIT: notepad shows the same characters as vs; ie, they both show the ñ. In the same place.).

2) vs2008 can't handle ñ. I find that very, very hard to believe.

3) The text is read in properly, but the default font for vs2008 can't display it, which is why the debugger shows a square.

4) The text is not read in properly, and I should use something other than a regular StreamReader to get strings.

5) The text is read in properly, but the default String class in C# doesn't handle ñ well. I find that very, very hard to believe.

6) The version of Arial Unicode MS I have doesn't have ñ, despite it being listed as one of the 50k characters by http://www.fileinfo.info.

Anything else I could have left out?

Thanks for any help!

Answer 1

I was having a similar problem just the other day - see Unicode characters not showing in System.Windows.Forms.TextBox. I was able to fix by changing a TextBox to a RichTextBox.

Answer 2

I would say that most certainly Notepad is the culprit. Notepad does not deal well with unicode characters. If you want to hand edit this file, use something like Notepad++ which can handle unicode, and make sure you save the file as UTF-8. You can probably just use VS to edit the file, and just forget about notepad or Notepad++ completely. .Net and Visual studio are actaully very good at handling accented characters. All strings are UTF-8 by default, so the problem almost certainly lies with Notepad.

Answer 3

Have you tried using String.Format when assigning the button.Text property, and providing the proper IFormatProvider with a spanish CultureInfo object?

I don't know if that would have an effect, but could help.

Answer 4

How are you reading the strings?

Have you tried to read the text file like this (with the encoding on set to UTF8):

using(StreamReader sr = new StreamReader(File.Open("file.txt", FileMode.Open), Encoding.UTF8))
{
// add your string to dictionary
}

Answer 5

I have a very short guide to debugging Unicode problems. It's targeted at fetching text from databases, but the same principles apply in general.

The most important starting point IMO is to know what's actually in your string when it just shows a box. Dump the contents to the console, with code like this:

static void DumpString (string value)
{
    foreach (char c in value)
    {
        Console.Write ("{0:x4} ", (int)c);
    }
    Console.WriteLine();
}

Then look up the character in the code charts on unicode.org. I suspect you want U+00F1, but there may be another similar character with a different code point - I've been fooled by that before.

Answer 6

Have you checked that your source file encoding is really utf-8? May not be applicable to default vs2008 install but IDE may detect your OS' default locale (or filesystem encoding) and set the matching non-utf-8 encoding for all your files. You might want to try with the doubly encoded mess (which you often come across on the web) "ñ" without changing anything in your setup to test your encoding mismatches.

I have bitten by this thing when I get to work on a coworker's god-knows-what-editor-in-what-encoding code.

I'm quite sure and assume all your api calls are utf-8 aware so all your text is interpreted as utf-8 even if it is not.

Answer 7

To read correctly spanish characters (ñ, á, é, etc) you can try the Codepage 1252 for the Encoding.