Ruby 1.9 doesn't support Unicode normalization yet

Question

I'm trying to port over some of my old rails apps to Ruby 1.9 and I keep getting warnings about how "Ruby 1.9 doesn't support Unicode normalization yet." I've tracked it down to this function, but I'm getting about 20 warning messages per request:

rails-2.3.5/activesupport/lib/active_support/inflector.rb

def transliterate(string)
  warn "Ruby 1.9 doesn't support Unicode normalization yet"
  string.dup
end

Any ideas how I should start tracking these down and resolving it?

Answer 1

add an

puts caller

into that method?

-r

Answer 2

That method definition is wrapped in an if-statement for Ruby 1.9. Right above it, you will find the regular definition, which shows a bit more of what this is doing. It's a method used to convert accented characters into their regular variants. E.g.: á => a, or ë => e

But this method is only used in parameterize, which is in turn defined right above transliterate. This is all still in ActiveSupport. I can't find anything that is directly calling parameterize.

So perhaps you're using parameterize or transliterate yourself, somewhere in your Rails application?

Common usage (according to the parameterize documentation) is for creating friendly permalinks from arbitrary strings, much like SO does, for example:

http://stackoverflow.com/questions/2135247/ruby-1-9-doesnt-support-unicode-normalization-yet
                                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Answer 3

Replace the body of the method with

raise "transliterate called"

and observe a backtrace which will show you where the stuff is coming from at the first call. Your app will of course collapse as well but that will likely give you the culprit from the first try.

Answer 4

I appreciate that this is a dirty way to solve the problem, but having read the error message I'm aware of the issue. So I want to get rid of the warnings. I dropped this code in environment.rb:

module ActiveSupport
  module Inflector
    # Calling String#parameterize prints a warning under Ruby 1.9,
    # even if the data in the string doesn't need transliterating.
    # Maybe Rails 3 will have fixed it...?
    if RAILS_GEM_VERSION =~ /^2\.3/
      undef_method :transliterate
      def transliterate(string)
        string.dup
      end
    end
  end
end

Answer 5

If you are aware of the consequences, i.e. accented characters will not be transliterated in Ruby 1.9.1 + Rails 2.3.x, place this in config/initializers to silence the warning:

# http://stackoverflow.com/questions/2135247/ruby-1-9-doesnt-support-unicode-normalization-yet
module ActiveSupport
  module Inflector
    # Calling String#parameterize prints a warning under Ruby 1.9,
    # even if the data in the string doesn't need transliterating.
    if Rails.version =~ /^2\.3/
      undef_method :transliterate
      def transliterate(string)
        string.dup
      end
    end
  end
end

Rails 3 does indeed solve this issue, so a more future-proof solution would be to migrate towards that.