Subject: Re: fastthread -- what is it? - msg#00033

On Thu, 4 Jan 2007 00:55:41 -0800
"Joe Van Dyk" <joevandyk-Re5JQEeQqe8AvxtiuMwx3w@xxxxxxxxxxxxxxxx> wrote:

> Hi,
>
> A quick search didn't explain what fastthread is, and how it relates
> to mongrel. Why would I want to install fastthread?

Hi Joe, long answer is that ruby has a bug in how the array works and how
threads are garbage collected. When the current Ruby thread locking primitives
operate on threads, they tend to store idle ones in arrays. As the existing
thread locking mechanisms pull the threads off the array they never get garbage
collected and that causes a nasty leak.

At the time I was testing this, me and Bradley Taylor narrowed the problem down
and produced these two scripts:

http://pastie.caboo.se/10194
http://pastie.caboo.se/10317

The first one shows the leak by using Mutex (with graph), the second showed
proper GC operation by using Sync (with graph). Still wasn't enough because
several high profile idiots on ruby-lang tried to claim the test was invalid
but couldn't offer a reason why the first script leaked and the second didn't.

Later, after much arguing, calling me an idiot, debating the merits of OS
memory allocation, and other stupidities, we find out that Eric Mahurin fixed
the whole problem a year ago and it was missed:

http://blade.nagaokgaut.ac.jp/cgi-bin/scat.rb/ruby/ruby-core/5861

Thankfully, there's smart cool people like mentalguy who can make improvements
and provide them as gems. What mentalguy did is re-create all of the Ruby
thread locking gear in a C extension called fastthread. With fastthread we can
provide a modest speed boost as well as fix the memory leaks Ruby has by just
including a gem. He's pushing for it to become the new thread locking api in
Ruby, so hopefully he can get it in without much fuss.

I'm telling people that you have to use fastthread, but it's still external so
if you run into problems or don't see an improvements then you don't have to
run it. Mongrel has fastthread as a gem dependency, but it will run it only if
it's installed. So, you can install it, try it out, and then remove the gem if
it's a problem.

And that's the whole story. Enjoy!

--
Zed A. Shaw, MUDCRAP-CE Master Black Belt Sifu
http://www.zedshaw.com/
http://www.awprofessional.com/title/0321483502 -- The Mongrel Book
http://mongrel.rubyforge.org/
http://www.lingr.com/room/3yXhqKbfPy8 -- Come get help.