ansaurus

Question

Bash equivalent to Python's string literal for utf string conversion.

Answer 1

+1 A:

possible solution, e.g.:

$ function conv() { echo $* | python -c 'import re, sys; print re.sub(r"&#(\d+);", lambda x: unichr(int(x.group(1))), sys.stdin.read()).rstrip()' ; }
$ conv '&#1040;&#1083;&#1080;&#1089;&#1040; (Alisa)' 
АлисА (Alisa)

mykhal 2010-07-07 01:28:03

if `UnicodeEncodeError` occurs, add `reload(sys); sys.setdefaultencoding("UTF-8");` after `import re, sys;`

mykhal 2010-07-07 01:40:08

Thanks! I'm still new to programming, and I wonder whether calling python or other languages causes a considerable use of system resources?

teratomata 2010-07-07 02:10:57

@teratomata yes, it is slow. launching of e.g. perl is considerably fater than python. but i would be still slower than if you could do it in bash (echo) directly

mykhal 2010-07-07 06:48:05

Answer 2

+1 A:

The printf builtin in Bash doesn't support Unicode codes, but the external printf (at least on my GNU-based system) does:

$ /usr/bin/printf "\u0410\u043b\u0438\u0441\u0410"
АлисА

or this, which selects printf from your path in case it's not in /usr/bin:

$ $(type -P printf) "\u0410\u043b\u0438\u0441\u0410"
АлисА

Dennis Williamson 2010-07-07 05:42:55

it's really working to you? the values must be converted to hex: `"\u0410\u043b\u0438\u0441\u0410"`

mykhal 2010-07-07 06:50:27

@mykhal: Oops, I pasted the wrong thing. Fixed, thanks.

Dennis Williamson 2010-07-07 07:20:50

Dennis Williamson: it's much better than sending a string to some python/perl/whatever one-liner

mykhal 2010-07-07 09:08:43

Thanks! I thought it shouldn't be as difficult as I thought but now that I'm looking into python it looks fairly attractive to learn.

teratomata 2010-07-07 16:39:52

Not to mention someone has already made modules to encode and decode html.

teratomata 2010-07-07 16:54:41

ansaurus

tags:

views:

answers:

Bash equivalent to Python's string literal for utf string conversion.

related questions