Re: ghc releasing memory during compilation

Bulat Ziganshin wrote:
> Sunday, March 12, 2006, 8:00:05 PM, you wrote:
>
> ghbrho> Subject: ghc releasing memory during compilation
>
> ghbrho> Am I right in thinking that ghc's rts can free memory back to the system
> ghbrho> when the heap pressure reduces (at least if it's doing a compacting GC)?
>
> ghbrho> In this case if it can do so, it should be quite dramatic. It'd ought to
> ghbrho> be able to go from 400Mb back down to just a few Mb or so.
>
> i've suggested the same just 11 months ago. after long discussion
> Simon Marlow declined my proposal because this will raise gc times in
> the situations when memory is enough by whole 10%! when i suggested
> just to check available PHYSICAL memory he answered that he don't know
> that is physical memory, only virtual memory matters

I think that's a mischaracterisation of what I said.  Actually I said 
that I didn't understand your comment about physical vs. virtual memory, 
and in fact, looking back at the message, I still don't understand it :)

http://www.haskell.org//pipermail/glasgow-haskell-users/2005-April/008373.html

I think what you're suggesting is that the runtime should detect the 
amount of physical memory on the system and auto-tune itself to switch 
to compacting collection when its residency reaches that amount.  This 
is certainly something we could do.  Bear in mind that GHC is not 
necessarily the only process running on the machine, though, and what 
about running multiple GHCs?

Also, you can do this by setting your GHC_RTS environment variable. 
Suppose you have 1G of physical mem.  You want GHC to give up when it 
reaches 1.5G, and you want to switch to compacting collection when the 
residency reaches 300M.  You could do this:

export GHC_RTS='-M1.5G -c20'

because 300M is 20% of 1.5G.  Perhaps a better interface would be to 
allow you to specify exactly the residency at which to switch to compaction.

Cheers,
        Simon