I'm trying to pass parameters from a PHP middle tier to a java backend that understands J2EE. I'm writing the controller code in Groovy. In there, I'm trying to decode some parameter that will likely contain international characters.
I am really puzzled by the results of my debugging this problem so far, hence I wanted to share it with you in the hope that someone will be able to give the correct interpretation of my results.
For the sake of my little test, the parameter I'm passing is "déjeuner". Just to be sure, System.out.println("déjeuner") correctly gives me:
déjeuner
in the console
Now following are the char/dec and hex values of each char of the original string:
next char: d 100 64
next char: ? -61 c3
next char: ? -87 a9
next char: j 106 6a
next char: e 101 65
next char: u 117 75
next char: n 110 6e
next char: e 101 65
next char: r 114 72
note that the c3a9 sequence in UTF-8 is the wished-for character: http://www.fileformat.info/info/unicode/char/00e9/index.htm
Now if I try to read this string as an UTF-8 string, as in stmt.getBytes("UTF-8"), I suddenly end up having a 11 bytes sequence, as follows:
64 c3 83 c2 a9 6a 65 75 6e 65 72
whereas stmt.getBytes("iso-8859-1") gives me 9 bytes:
64 c3 a9 6a 65 75 6e 65 72
note the c3a9 sequence here!
now if I try to convert the UTF-8 sequence to UTF-8, as in
new String(stmt.getBytes("UTF-8"), "UTF-8");
I get:
next char: d 100 64
next char: ? -61 c3
next char: ? -87 a9
next char: j 106 6a
next char: e 101 65
next char: u 117 75
next char: n 110 6e
next char: e 101 65
next char: r 114 72
note the c3a9 sequence
while
new String(stmt.getBytes("iso-8859-1"), "UTF-8")
results in:
next char: d 100 64
next char: ? -23 e9
next char: j 106 6a
next char: e 101 65
next char: u 117 75
next char: n 110 6e
next char: e 101 65
next char: r 114 72
note the e9 which in utf-8 (and ascii) is, again, the 'é' character that I'm longing for.
Unfortunately, in neither case am I ending up with a proper string that would display like the literal string "déjeuner". Strangely enough, the byte sequences both seem correct though.