Replace national characters with ASCII equivalent.

Question

Is there any lib that can replace national characters to ASCII equivalents, like:

"Cześć"

to:

"Czesc"

I can of course create map:

{'ś':'s', 'ć': 'c'}

and use some replace function. But I don't want to hardcode all equivalents into my program, if there is some function that already does that.

Answer 1

#!/usr/bin/env python
# -*- coding: utf-8 -*-

import unicodedata
text = u'Cześć'
print unicodedata.normalize('NFD', text).encode('ascii', 'ignore')

Answer 2

You can get most of the way by doing:

import unicodedata

def strip_accents(text):
    return ''.join(c for c in unicodedata.normalize('NFKD', text) if unicodedata.category(c) != 'Mn')

Unfortunately, there exist accented Latin letters that cannot be decomposed into an ASCII letter + combining marks. You'll have to handle them manually. These include:

• Æ → AE
• Ð → D
• Ø → O
• Þ → TH
• ß → ss
• æ → ae
• ð → d
• ø → o
• þ → th
• Œ → OE
• œ → oe
• ƒ → f

Answer 3

The unicodedata.normalize gimmick can best be described as half-assci. Here is a robust approach which includes a map for letters with no decomposition. Note the additional map entries in the comments.