Subject: Re: Rubyforge gem sync frequency - msg#00066

Previous Message by Date: click to view message preview

Re: pure-ruby vs. compile-on-intsall

On 7 Nov 2007 13:37:30 +0100, Florian Frank <flori-PLamJo5MY9R6lmGzAMPh1A@xxxxxxxxxxxxxxxx> wrote: > Trans wrote: > >>>> I'm a bit confused. If I have a pure-ruby version of my lib, but also > >>>> have some extensions that can speed things up, should the pure-ruby > >>>> gem be named plainly? Eg. 'foo-1.0.0.gem'. But then what platform is > >>>> the compile-on-install gem? Am I going about this wrong? I'm starting > >>>> to think it would be easier to create two packages, one for the > >>>> pure-ruby gem and another for the optional extensions. > >>>> > > > > I need to provide these: > > > > * pure ruby version > > * compile-on-install version > > * win32 pre-built version > > > > I would like to make the first two a single gem, rather then two > > separate gems. To do that I need RubyGems not to abort if make fails. > > I am going to try to figure out a way to do that. But if someone > > already has an idea how, please let me know. > > > If you figure this out, please post the solution to the list. I had/have > exactly the same problem with my json gem, which offers a pure ruby > implementation variant and a faster c extension variant. > > I have used another name (json_pure) for the pure ruby variant, which > solves one problem: You now can pick the json variant, if your platform > supports compilation, and you can pick json_pure if you have to use the > pure version. Ironically my json gem would automatically fallback to the > pure variant anyway, if the shared objects couldn't be loaded. So the > json_pure variant contains almost the same files as the json variant > does, and the json variant includes the pure ruby version, which can be > used by requiring 'json/pure' instead of requiring 'json'. > > Another problem isn't solved by this workaround, though. If some other > developer adds json as a dependency, it will resolve to the compile > variant, but the resolution will fail altogether on a platform like > jruby, which doesn't support compilation. > > I have learned that merb ran into this problem, they now use json_pure > as a dependency. This is far from optimal. I have made made a RubyGems > feature request for a pure ruby platform > (http://rubyforge.org/tracker/index.php?func=detail&aid=13290&group_id=126&atid=578) > a while ago, in order to address this problem, but it was closed without > a real solution. > > It's unfortunate, that RubyGems conflates gems, that are pure ruby with > those that will trigger a compilation process, but I guess it's too late > to change this now. In the case of json, it would be indeed be enough, > if RubyGems would soldier on and just install the pure ruby stuff > anyway, if compilation fails. But this isn't guaranteed to work for > other gems, that don't have a pure ruby fallback. > > The gem command could do a better job to resolve dependencies as well: > > It could first figure out, if the platform under which is running, > supports compilation. If the json dependency is being resolved on > platform like jruby, that doesn't support compilation like jruby, it > should first look for a binary platform specific gem, e. g. jruby > platform. If none is found, it should look for a pure platform variant > and try to install that one. If there is no pure platform found, it > should assume, that the ruby platform gem is actually a pure ruby > implementation and try to install that one. This could still fail, but > this would be the same behaviour, that is happening now. It can get tricky. Gems really needs to offer a switch as well so the installer can select the variant. I managed a partial solution. There did not seem to be a way to rescue the compile failure b/c Rubygems is shelling out the execution of extconf.rb. So instead I made prints a message when compiling fails that tells the user they can still use the library in pure-ruby mode by setting an environment variable and installing again. My extconf.rb looks like this: require 'mkmf' require 'rbconfig' extension_name = 'base64' arch = Config::CONFIG['arch'] FailedMessage.replace("Could not create Makefile, probably for the lack of necessary libraries and/or headers. Check the mkmf.log file for more details. You may need configuration options (see below). TMail has a pure-ruby fallback mode, so you can still use this library. To do so, set the environment variable, export NORUBYEXT='true', and gem install again.\n\n") if ENV['NORUBYEXT'] == 'true' # Rubygems is sending all output to dev/null :( STDOUT << "NORUBYEXT option is set to true. Native extension will be omitted." File.open('Makefile', 'w') do |f| f << "all:\n" f << "install:\n" end else if (/mswin/ =~ RUBY_PLATFORM) and ENV['make'].nil? $LIBS += " msvcprt.lib" create_makefile(extension_name, "tmail/#{arch}") else $CFLAGS += " -D_FILE_OFFSET_BITS=64" #??? create_makefile(extension_name, "tmail/#{arch}") end end As you can see I just made a dummy Makefile that does nothing if RUBYNOEXT is set to 'true' (Hmm... I should probably give the variable a mode specific name, anyway...) You'll also notice I use 'arch' for the lib destination. I use a #require_arch method in my code to first attempt to load the native code: def require_arch(fname) arch = Config::CONFIG['arch'] begin path = File.join("tmail", arch, fname) require path rescue LoadError => e # try pre-built Windows binaries if arch =~ /mswin/ require File.join("tmail", 'mswin32', fname) else raise e end end end I'm still running it through some paces but it seems to be working fine. It's not the ideal solution, but it's workable for now. The ideal solution would be for RubyGems to add a 'ruby_ext_fallback' option to the gemspec. I have a quick patch if the RubyGem team is interested. I believe it's as simple adding an if @spec.ruby_ext_fallback to Installer#build_extensions method. T.

Next Message by Date: click to view message preview

Platform-specific gems not getting picked up in the index

Hi all - Something seems to have changed in the indexer; only certain gems are getting picked up. For example, for wxruby we now see only: ========================== $ gem install wxruby Select which gem to install for your platform (i686-linux) 1. wxruby 1.9.2 (x86_64-linux) 2. wxruby 1.9.1 (x86_64-linux) 3. Cancel installation ========================== But there are a bunch of gems for different platforms available: ========================== $ ls /var/www/gems/gems/ | grep wxruby- wxruby-1.9.0-i386-mswin32.gem wxruby-1.9.0-i686-darwin8.4.1.gem wxruby-1.9.0-i686-linux.gem wxruby-1.9.0-powerpc-darwin8.10.0.gem wxruby-1.9.1-i386-mswin32.gem wxruby-1.9.1-i686-darwin8.4.1.gem wxruby-1.9.1-i686-linux.gem wxruby-1.9.1-powerpc-darwin8.3.0.gem wxruby-1.9.1-x86_64-linux.gem wxruby-1.9.2-i386-mswin32.gem wxruby-1.9.2-i686-darwin8.8.2.gem wxruby-1.9.2-i686-linux.gem wxruby-1.9.2-powerpc-darwin8.10.0.gem wxruby-1.9.2-x86_64-linux.gem ========================== Eric, maybe it's only getting the last gem for each name - something like that? Thanks, Tom

Previous Message by Thread: click to view message preview

Re: Rubyforge gem sync frequency

On Nov 7, 2007 2:30 AM, Luis Lavena <luislavena-Re5JQEeQqe8AvxtiuMwx3w@xxxxxxxxxxxxxxxx> wrote: > On 11/7/07, Trans <transfire-Re5JQEeQqe8AvxtiuMwx3w@xxxxxxxxxxxxxxxx> wrote: > > On Nov 6, 2007 9:58 PM, Luis Lavena > > <luislavena-Re5JQEeQqe8AvxtiuMwx3w@xxxxxxxxxxxxxxxx> wrote: > > > On 11/6/07, Trans <transfire-Re5JQEeQqe8AvxtiuMwx3w@xxxxxxxxxxxxxxxx> > > > wrote: > > > > How long does it take to refresh the rubyforge gem index? I put a gem > > > > up 20mins ago and gem is still telling me it's not there: > > > > > > > > > > You should as RubyForge staff about that (Tom?). > > > > That's Tom Copeland. Not me. But Tom Copeland is a great guy :-) > > > > I missed the 'k' ... you should ask :-) Hahahaha. And I thought you had really botched that sentence :-) Ahhh... what am I doing up at this hour? Lol. T.

Next Message by Thread: click to view message preview

Re: Rubyforge gem sync frequency

On Nov 7, 2007 9:13 AM, Tom Copeland <tom-rlPNtO1eV9h54TAoqtyWWQ@xxxxxxxxxxxxxxxx> wrote: > On Tue, 2007-11-06 at 23:58 -0300, Luis Lavena wrote: > > On 11/6/07, Trans <transfire-Re5JQEeQqe8AvxtiuMwx3w@xxxxxxxxxxxxxxxx> wrote: > > > How long does it take to refresh the rubyforge gem index? I put a gem > > > up 20mins ago and gem is still telling me it's not there: > > > > > > > You should as RubyForge staff about that (Tom?). > > We've got a cronjob that checks for new gems once an hour. Thanks. Good to know. T.