Re: ActiveSupport::Multibyte for better Unicode support

On 27/09/2006, at 4:48 PM, Manfred Stienstra wrote:

> On Sep 27, 2006, at 2:27 AM, Pete Yandell wrote:
>>
>> As I said in an earlier email, the laundry list reads something like:
>> - Make sure your database character set is utf8 <- this should
>> possibly be checked by Rails
>
> Like someone said before, setting your database encoding to utf-8 is
> only important if you want to do string operations. Otherwise you can
> just use the database as a bitbucket and it won't matter. I think
> this this should be the default in railties and not handled by a  
> plugin.

I vehemently disagree! :) If you're storing UTF8 data, you shouldn't  
have your database think it's latin1. (You can only get away with  
this at all because MySQL is lazy with checking strings.) Backing up,  
exporting, or accessing the database through something other than  
Rails can all give you trouble if you do this.

>> - Make sure all your tables have a character set of utf8 <- this
>> should be done in migrations
>
> The best solution is to set the default encoding of the database when
> you create it, that way you can't miss a table and you still have the
> option to override it for certain tables.

I agree, but it would be nice to have Rails at least say "I think I'm  
running in utf8 mode, so I'd better make sure the database agrees and  
warn the developer if it doesn't."

>> - Add 'normalize_unicode_params :form => :kc' to your application.rb
>
> Compatability normalization should _never_ be a default, because it
> causes data loss. If there is a default, it should probably be NFC or
> NFD. I'm still not convinced it's important to normalize all incoming
> data.

Yep, fair call. I think this is the trickiest point.

> I am in no way trying to stop you from writing your own plugin, but I
> hope you don't waste time going down the same route we did.

Well, if nothing else my plugin will be useful to me. I'm sick of  
having to go through all the steps required to support unicode in  
every app I write, and I've accidentally missed steps before in ways  
that have caused nasty data corruption and been hard to fix. (Try  
setting your database character set to utf8, but not setting the  
connection character set.)

Pete Yandell

--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups "Ruby 
on Rails: Core" group.
To post to this group, send email to rubyonrails-core@xxxxxxxxxxxxxxxx
To unsubscribe from this group, send email to 
rubyonrails-core-unsubscribe@xxxxxxxxxxxxxxxx
For more options, visit this group at 
http://groups.google.com/group/rubyonrails-core
-~----------~----~----~----~------~----~------~--~---