Re: fastthread -- what is it?

On Thu, 4 Jan 2007 00:55:41 -0800
"Joe Van Dyk" <joevandyk-Re5JQEeQqe8AvxtiuMwx3w@xxxxxxxxxxxxxxxx> wrote:

> Hi,
> 
> A quick search didn't explain what fastthread is, and how it relates
> to mongrel.  Why would I want to install fastthread?

Hi Joe, long answer is that ruby has a bug in how the array works and how 
threads are garbage collected.  When the current Ruby thread locking primitives 
operate on threads, they tend to store idle ones in arrays.  As the existing 
thread locking mechanisms pull the threads off the array they never get garbage 
collected and that causes a nasty leak.

At the time I was testing this, me and Bradley Taylor narrowed the problem down 
and produced these two scripts:

http://pastie.caboo.se/10194
http://pastie.caboo.se/10317

The first one shows the leak by using Mutex (with graph), the second showed 
proper GC operation by using Sync (with graph).  Still wasn't enough because 
several high profile idiots on ruby-lang tried to claim the test was invalid 
but couldn't offer a reason why the first script leaked and the second didn't.

Later, after much arguing, calling me an idiot, debating the merits of OS 
memory allocation, and other stupidities, we find out that Eric Mahurin fixed 
the whole problem a year ago and it was missed:

http://blade.nagaokgaut.ac.jp/cgi-bin/scat.rb/ruby/ruby-core/5861

Thankfully, there's smart cool people like mentalguy who can make improvements 
and provide them as gems.  What mentalguy did is re-create all of the Ruby 
thread locking gear in a C extension called fastthread.  With fastthread we can 
provide a modest speed boost as well as fix the memory leaks Ruby has by just 
including a gem.  He's pushing for it to become the new thread locking api in 
Ruby, so  hopefully he can get it in without much fuss.

I'm telling people that you have to use fastthread, but it's still external so 
if you run into problems or don't see an improvements then you don't have to 
run it.  Mongrel has fastthread as a gem dependency, but it will run it only if 
it's installed.  So, you can install it, try it out, and then remove the gem if 
it's a problem.

And that's the whole story.  Enjoy!

-- 
Zed A. Shaw, MUDCRAP-CE Master Black Belt Sifu
http://www.zedshaw.com/
http://www.awprofessional.com/title/0321483502 -- The Mongrel Book
http://mongrel.rubyforge.org/
http://www.lingr.com/room/3yXhqKbfPy8 -- Come get help.