Display problem with Japanese characters

Question

I am fetching a Japanese string from Oracle Database and displaying it on the browser. But the characters are shown on the browser like ???. Inserted the Japanese string into DB using the unistr() function.

INSERT INTO MESSAGES (MESSAGE_ID,MESSAGE) VALUES (1,unistr('\0041\0063\0063\0065\0073\0073\0020\004d\0061\006e\0061\0067\0065\006d\0065\006e'));    

I got this in my jvm logs ISO8859-1 when I printed System.getProperty("file.encoding").

select * from v$nls_parameters where parameter in ('NLS_CHARACTERSET') yields UTF8 in my DB.

Any pointers on how the Japanese characters could be displayed correctly ?

Answer 1

Try changing your browser locale. You can do it with Firefox.

Answer 2

The character of the server's response to the browser appears to be incorrectly specified as far as Unicode encoding is concerned. There is a fairly detailed SO question on this topic, if you're using Tomcat. Do note that you have to use UTF-8 and not ISO-Latin-1/ISO-8859-1 as Japanese characters do not fall into the Latin-1 character encoding.

In addition to the pointers about Unicode encoding, you might want to check a couple of things:

• Is the data in the database stored in the manner you desire? You could use a tool like Oracle SQL Developer (which has Unicode support by default; I'm not sure about Japanese fonts, but you could switch to them) to view the contents of the database tables.
• Are you setting the correct encoding for the application server JVM?
• Are you viewing the pages with a Japanese font installed for the browser, and with the Japanese locale?

If you've ruled out all of the above, then Unicode characters (including Japanese) are being converted into a format that is not understandable.

Answer 3

Checklist:

1. Check your browser locale (as suggested by duffymo).
2. Do you have the Asian Fonts installed (if running windows). Browse to www.yahoo.co.jp -- can you see Japanese characters here?
3. If you have the Japanese fonts and Japanese locale, try running the following statement directly :

select unistr('\0041\0063\0063\0065\0073\0073\0020\004d\0061\006e\0061\0067\0065\006d\0065\006e') from dual

If #3 is successful, then the file encoding of the message field in your table is incorrect.

Answer 4

If you are seeing ??? in the webbrowser, then changing the browser's locale/charset as suggested by others really won't help much. Only if you were seeing �, empty squares and/or Mojibake, then it may indeed help. Also installing fonts really won't help much as well. If there wasn't a font for it, you would in Firefox have seen squares with hexcodes inside and in IE empty squares and really not ???.

The ??? can here have only one cause: you are writing those characters to the HTTP response using the wrong encoding. The average webserver will replace unknown characters by ?. The webbrowser doesn't do that, it is just displaying them as is. Actually, there is in theory another possible cause; the DB will do the same when you're inserting unknown characters, but that's less or more excluded here.

It's not clear what view technology you're using, but since you're talking about Java and a webbrowser, I'll assume that you're using JSP/Servlet (in the future, please mention and tag as such, so that the right audience will be reached).

If you're displaying those characters using JSP, then you need to add the following to the top of your JSP page to instruct the servletcontainer to write those characters using the right encoding:

<%@ page pageEncoding="UTF-8" %>

If you're writing those characters manually using a Servlet, then you need to set the HTTP servlet response to use the right encoding as follows before you write any character to it:

response.setCharacterEncoding("UTF-8");

See also:

• Unicode - How to get the characters right?