Encoding problem undr Groovy

Question

Hi! I am developing an application which should work under different languages (german, spanish, etc). The application uses an Oracle DB. I have CRUD business processes and I have the standard views (create, edit, show, list). No problems for creating, showing and listing any record containing special chars like ä,ö,ü, etc. But when I edit any entry containing any of this characters I am getting the encoded version. i.e. & auml; instead of ä & ouml; instead of ö & uuml; instead of ü

and so on.

Any hint how to solve this problem?

Thanks!

UPDATE Thanks for your help. I will describe the complete scenario:

I have a web application written in grails (groovy on grails). For developing I am using Jetty as server and Oracle 10g. For testing and production I am using Tomcat 6.0.18 and Oracle 10g Java version is 1.6.0_02

I have many CRUD processes (create, retrieve, update, delete). The application is multilanguage. That is, ä, ö, ü, ß, á, é, í, ó, ú letters (characters) must be allowed as content.

Views are written in gsp. I am using standard .gsp views (create, edit, show, list). No problem with create, show, list. That is, if under the create view I type any word using this special characters then this will be shown or listed correctly under the show.gsp or list.gsp

The problem arises when editing a record containing such characters. Instead of -let's say- ä appears in the field an & auml; (this is the html encoding of ä).

I have the following settings:

under Config.groovy grails.views.gsp.encoding="UTF-8" grails.converters.encoding="UTF-8"

Every .gsp page has the following meta tag:

 <meta http-equiv="Content-Type" content="text/html; charset=UTF-8" pageEncoding="UTF-8"/>

Every form has the following attribute:

<g:form accept-charset="UTF-8" method="post" >

Under Tomcat I did the following settings.

tomcat is started with the following options:

CATALINA_OPTS=-Dfile.encoding=UTF-8
JAVA_OPTS="-Duser.language=de -Duser.country=DE"

Under web.xml I set the following filter

 <filter>
   <filter-name>SetCharacterEncoding</filter-name>
   <filter-class>filters.SetCharacterEncodingFilter</filter-class>
   <init-param>
       <param-name>encoding</param-name>
       <param-value>UTF-8</param-value>
   </init-param>
 </filter>

Under myApplication/WEB-INF/classes/filters I copied the SetCharacterEncodingFilter.class from examples/WEB-INF/classes/filters

Under server.xml I set the following connector:

 <Connector port="8080" protocol="HTTP/1.1"
           connectionTimeout="20000"
           redirectPort="8443" URIEncoding="UTF-8" useBodyEncodingForURI="UTF-8" />

The scenario is as follows: The server receives a request for editing a form. The server is retrieving the info from the DB and then either the DB is sending the information already html encoded (I don't think so) or the server is encoding it and sending it encoded to the client.

Moreover, at my controller I can see that the retrieved information from the server is not html encoded.

I don't know which setting has to be done in order to cope this encoding problem (which is taking me a lot, a lot of time and effort).

Thanks a lot in advance.

Luis

Answer 1

encoding is a tough problem because you can't always trust what you see. Usually using utf-8 is enough to solve the problem. Make sure you're using it everywhere, though - oracle server, oracle client (jdbc), groovy files, etc. Be aware that java uses utf-16. The jdbc driver should do this conversion for you. The best advice to give is that java (and thus groovy) do exactly what they say they do when it comes to character encoding so the problem isn't there. Make sure you have tools that let you examine the actual bits (hex value) for each aspect in the system. In mysql - it's just hex: http://dev.mysql.com/doc/refman/5.0/en/string-functions.html#function_hex.
I assume oracle has a similar function. Again make sure your client is configured to use utf-8. I got burned by that before. To examine source files (or log files) use something like xxd or od. If you're using cygwin, make sure you turn on utf8 (in the vt fonts menu). If you're on windows and not using cygwin, you should definitely take it for a spin (make sure you use the x version).

Answer 2

Besides all the settings already done. I've set:

grails.views.default.codec="html" // none, html, base64

instead of none and the problem is solved.

My problem was already exposed here: http://www.groovy-forum.de/read.php?3,5008,5630

For more information see:

http://jira.codehaus.org/browse/GRAILS-1827

Luis