Determining if a Unicode character is visible?

Question

I am writing a text editor which has an option to display a bullet in place of any invisible Unicode character. Unfortunately there appears to be no easy way to determine whether a Unicode character is invisible.

I need to find a text file containing every Unicode character in order that I can look through for invisible characters. Would anyone know where I can find such a file?

EDIT: I am writing this app in Cocoa for Mac OS X.

Answer 1

What platform are you using? There may be a call which would help, e.g. in .NET there's Char.GetUnicodeCategory.

Answer 2

Its an impossible task, Unicode supports even Klingon, so it's not going to work. However most text editors use the standard ANSI invisible characters. And if your Unicode library is good, it will support finding equivalent characters and/or categories, you can use these two features to do it as well as any editor out there

Edit: Yes I was being silly about Klingon support, but that doesn't make it not true... of course Klingon is not supported by the Consortium, however there is a movement for Klingon in the Unicode's "Private Use Area" defined for Klingon alphabet (U+F8D0 - U+F8FF). Link here for those interested :)

Note: Wonder what editor Klingon programmers use...

Answer 3

A good place to start is the Unicode Consortium itself which provides a large body of data, some of which would be what you're looking for.

I'm also in the process of producing a DLL which you give a string and it gives back the UCNs of each character. But don't hold your breath.

Answer 4

The current official Unicode version is 5.1.0, and text files describing all of the code points in that can be found at http://www.unicode.org/standard/versions/components-latest.html

Answer 5

Oh, I see... actual invisble characters ;) This FAQ will probably be useful:

http://www.unicode.org/faq/unsup_char.html

It lists the current invisible codepoints and has other information that you might find helpful.

EDIT: Added some Cocoa-specific information

Since you're using Cocoa, you can get the unicode character set for control characters and compare against that:

NSCharacterSet* controlChars = [NSCharacterSet controlCharacterSet];

You might also want to take a look at the FAQ link I posted above and add any characters that you think you may need based on the information there to the character set returned by controlCharacterSet.

EDIT: Added an example of creating a Unicode string from a Unicode character

unichar theChar = 0x000D;
NSString* thestring = [NSStirng stringWithCharacters:&theChar length:1];

Answer 6

For Java, java.lang.Character.getType. For C, u_charType() or u_isgraph().

Answer 7

How can I convert a Unicode value into the character itself? For example, 000D.

Answer 8

you might find this code to be of interest: http://gavingrover.blogspot.com/2008/11/unicode-for-grerlvy.html

Answer 9

Let me know if this code helps at all:

-(NSString*)stringByReplacingControlCharacters:(NSString*)originalString
{
    NSUInteger length = [originalString length];
    unichar *strAsUnichar = (unichar*)malloc(length*sizeof(unichar));
    NSCharacterSet* controlChars = [NSCharacterSet controlCharacterSet];
    unichar bullet = 0x2022;

    [originalString getCharacters:strAsUnichar];
    for( NSUInteger i = 0; i < length; i++ ) {
     if( [controlChars characterIsMember:strAsUnichar[i]] )
      strAsUnichar[i] = bullet;
    }

    NSString* newString = [NSString stringWithCharacters:strAsUnichar length:length];
    free(strAsUnichar);

    return newString;
}

Important caveats:

This probably isn't the most efficient way of doing this, so you will have to decide how you want to optimize after you get it working. This only works with characters on the BMP, support for composted characters would have to be added if you have such a requirement. This does no error checking at all.