normalize-text

Programatic Accent Reduction in JavaScript (aka text normalization or unaccenting)

I need to compare 2 strings as equal such as these: Lubeck == Lübeck In JavaScript. Why? Well, I have an auto-completion field that's going out to a Java service using Lucene, where place names are stored naturally (as Lübeck), but also indexed as normalized text, import sun.text.Normalizer; oDoc.setNameLC = Normalizer.normalize...

How do I properly implement Unicode passwords?

Adding support for Unicode passwords it an important feature that should not be ignored by developers. Still, adding support for Unicode in passwords is a tricky job because the same text can be encoded in different ways in Unicode and you don't want to prevent people from logging in because of this. Let's say that you'll store the pa...

tackle different types of utf hyphens in ruby 1.8.7

We have different types of hyphens/dashes (in some text) populated in db. Before comparing them with some user input text, i have to normalize any type of dashes/hyphens to simple hyphen/minus (ascii 45). The possible dashes we have to convert are: Minus(−) U+2212 − or − or − Hyphen-minus(-) U+002D - Hyphen(-) ...