Properly handling unicode characters in Rails

Question

By default Rails allows users of our application to input non-utf8 data, such as: ¶®«¼

However when we attempt to retrieve the data from our database and render it in a template Rails incorrectly assumes that it is in UTF-8 format and throws an error.

ArgumentError: invalid byte sequence in UTF-8

What is the best way to handle this? I have seen one fix that suggested sanitizing the data in every place the user can input it. However, that would involve changing a considerable amount of code and it would strip out the characters entirely. Ideally we would want some characters converted to their UTF-8 equivalents.

Our environment:

Ruby: 1.9.1
Rails 2.3.5
MySql Gem: 2.8.1

This is a serious and urgent problem for us so your answers are very appreciated!

Answer 1

Try using the mysql2 gem (version 0.1.6 or greater) instead of mysql.

It's still fairly new, but has been working well for me in production and will retrieve all strings from your DB as utf-8.

Just install the gem and change your database.yml file to contain "adapter: mysql2"