Re: ActiveSupport::Multibyte for better Unicode support

On 9/20/06, Manfred Stienstra <manfred@xxxxxxxxx> wrote:
>
> Three months ago Julian Tarkhanov submitted a test implementation of
> his ActiveSupport::Multibyte string extension patch. Since then we've
> been steadily improving the extension based on the feedback we received.

I'm studying it now...a few notes as I go and thoughts at the end:

- I think we could support Chars natively under JRuby pretty easily,
though everything would be UTF-16 internally. However Java has many,
many utilities already present for converting UTF-16 to damn near
every encoding under the sun, so this wouldn't be a real limitation.
Native JRuby support for MultiByte could potentially be significantly
faster than a pure Ruby version, but fully API-equivalent.
- We've been kicking around the possibility of migrating to a mutable
UTF-8 string inside JRuby, to avoid the wasted high byte and to get
all our mutability in a single type that's friendlier than what Java
provides. If we could say that the base JRuby String implementation
supports a fast, solid UTF-8 backing store normally and MultiByte's
String#chars and Chars for actual multibyte operations I think we'd
have the best of all worlds

I like the interface I'm seeing so far. The separation of the base
"dumb" String from the "smart" multibyte-aware Chars seems to be the
path of least resistance. In my opinion, it's potentially the "right
way" long term too...let String remain a dumb byte-box and provide a
character-aware type that knows how to do the "right thing" with
multibyte encodings.

I also think you guys are going to drive unicode adoption in Ruby for
the next year. Matz's m17n is a long way out, and people need unicode
now. If something like MultiByte gains serious traction, there's going
to be a lot of pressure to support that API in the long term, and
there would be little reason we couldn't support it out of the box in
core JRuby right now.

-- 
Contribute to RubySpec! @ www.headius.com/rubyspec
Charles Oliver Nutter @ headius.blogspot.com
Ruby User @ ruby.mn

--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups "Ruby 
on Rails: Core" group.
To post to this group, send email to rubyonrails-core@xxxxxxxxxxxxxxxx
To unsubscribe from this group, send email to 
rubyonrails-core-unsubscribe@xxxxxxxxxxxxxxxx
For more options, visit this group at 
http://groups.google.com/group/rubyonrails-core
-~----------~----~----~----~------~----~------~--~---