views:

51

answers:

3

I'm new to learning Unicode, and not sure how much I have to learn based on my ASCII background, but I'm reading the C# spec on rules for identifiers to determine what chars are permitted within Azure Table (which is directly based on the C# spec).

Where can I find a list of Unicode characters that fall into these categories:

  • letter-character: A Unicode character of classes Lu, Ll, Lt, Lm, Lo, or Nl
  • combining-character: A Unicode character of classes Mn or Mc
  • decimal-digit-character: A Unicode character of the class Nd
  • connecting-character: A Unicode character of the class Pc
  • formatting-character: A Unicode character of the class Cf
+1  A: 

FileFormat.info has a list of Unicode characters by category:

http://www.fileformat.info/info/unicode/category/index.htm

Phil Ross
+1 - This option renders best for casual browsing
MakerOfThings7
+2  A: 

You can, of course, use LINQ:

var charInfo = Enumerable.Range(0, 0x110000)
                         .Where(x => x < 0x00d800 || x > 0x00dfff)
                         .Select(char.ConvertFromUtf32)
                         .GroupBy(s => char.GetUnicodeCategory(s, 0))
                         .ToDictionary(g => g.Key);

foreach (var ch in charInfo[UnicodeCategory.LowercaseLetter])
{
    Console.Write(ch);
}

You can find a list of Unicode categories and their short names on MSDN, e.g., "Ll" is short for UnicodeCategory.LowercaseLetter.

dtb
How did you know to hard code those constants in? Where do they come from?
MakerOfThings7
@MakerOfThings7: From the documentation of [Char.ConvertFromUtf32](http://msdn.microsoft.com/en-us/library/system.char.convertfromutf32.aspx). It throws an exception if its argument "is not a valid 21-bit Unicode code point ranging from U+0 through U+10FFFF, excluding the surrogate pair range from U+D800 through U+DFFF."
dtb
Linq is fun. +1 since I'm going to learn something from this. Also I think not all chars will render within "Console.write". Perhaps it's better for me to output these codes in a HTML page for IE to render?
MakerOfThings7
@MakerOfThings7: Yes, the set of characters the Console can display is quite limited. Writing the characters to a HTML page is a good idea.
dtb
+1  A: 

You can retrieve this information in an automated fashion from the official Unicode data file, UnicodeData.txt, which is published here:

This is a file with semicolon-separated values in each line. The third column tells you the character class of each character.

The benefit of this is that you can get the character name for each character, so you have a better idea of what it is than by just looking at the character itself (e.g. would you know what ბ is? That’s right, it’s Ban. In Georgian. :-))

Timwi
Nice! I can even search for chars within each category like this ";Cf;"
MakerOfThings7
...I never in my life thought Unicode was this complex. Seems like I have a lot of learning to do.
MakerOfThings7