Re: ActiveSupport::Multibyte for better Unicode support

On 27/09/2006, at 11:51 AM, Michael Koziarski wrote:

>> As I said in an earlier email, the laundry list reads something like:
>> - Make sure your database character set is utf8 <- this should
>> possibly be checked by Rails
>> - Make sure all your tables have a character set of utf8 <- this
>> should be done in migrations
>> - Make sure your database.yml has 'encoding: utf8' set for each  
>> database
>
> We can't change these without the users intervention, and doing utf-8
> with postgres is a little harder than just 'setting the encoding' for
> the table.  Perhaps this is just something we need to include in our
> documentation?

We can certainly make sure tables created with migrations have the  
right character set, and we can at least check and give a warning if  
the various character sets (database, table, connection, Rails) don't  
match up.

I don't know what's required for Postgres, but I'll build for MySQL  
and somebody with Postgres experience can extend from there.

>> - Add 'normalize_unicode_params :form => :kc' to your application.rb
>
> Why do we need this?   I can understand the rationale for doing iconv
> for 'differently encoded' strings, but can't quite follow the
> justification of normalization.

Because there are characters in unicode (for example 'ü') that can be  
encoded in multiple possible ways (either as a single 'ü' character,  
or as a 'u' followed by an umlaut modifier), and without  
normalisation comparisons between them will fail.

This is probably the trickiest area. When and how to normalise is  
something a developer really needs to think about, so just enabling  
auto-normalisation of all parameters is possibly not the solution.

Pete Yandell
--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups "Ruby 
on Rails: Core" group.
To post to this group, send email to rubyonrails-core@xxxxxxxxxxxxxxxx
To unsubscribe from this group, send email to 
rubyonrails-core-unsubscribe@xxxxxxxxxxxxxxxx
For more options, visit this group at 
http://groups.google.com/group/rubyonrails-core
-~----------~----~----~----~------~----~------~--~---