Re: Useless kernel caching is annoying me

On Sun, Sep 29, 2002 at 01:50:08PM +0300, Nadav Har'El wrote:
> Hi people, I have a question that has been bugging me for quite some time
> now.

Very unscientific answers and recommendations follow. Caveat emptor. 

> I'm using Redhat 7.3 and its 2.4.18-3 kernel. I have 128MB of RAM. I
> frequently use memory-hogging applications such as Mozilla and
> OpenOffice.

That's an old an buggy kernel. At the very least, you should update to
redhat's latest errata kernel. 

> The phenomenon that annoys me is that many times, after I've left one of
> these applications for a bit of time (say, an hour), returning to it requires
> a huge amount of swapping and sometimes as much as 10 seconds of
> wait.

That's a lot of time... is it possible that your disks are slow, for
some reason? Are you using a swap partition or file? What does hdparm
say about your disks? 

> I have a suspicion (based on not-very-scientific experiments) that what is
> displacing these applications from memory is my mp3 playing habit. I'm
> constantly playing mp3 files (using xmms), and I suspect that the kernel
> is caching the files I'm playing in its caches.

That's certainly possible. 

> Further "evidence" is that "free" normally shows me about 50MB of "cached"
> on this machine, and "iostat" shows me swap-out activity once in a while,
> closely correlated with the time of disk *reads* (supposedly the time XMMS
> reads more of the mp3 file).
> 
> Does this make any sense? Does the kernel agree to swap out memory just
> to make place for a file that is going to be read once (after all, I won't
> play the same file again until about a week will pass)? 

The kernel has no way of knowing that that file (or more precisely,
its pages in memory) are going to be used only once. A lot of the VM
tuning efforts go towards establishing which pages are going to be
read only once (and are therefore excellent candidates for swap out /
not caching) and which are going to be used more often. 

> Or maybe I'm
> misunderstanding the problem and it's just that 2.4.18's VM sucks or
> something?

That's certainly possible... 

> If indeed the caching is hurting me, does anybody know any way to control
> this useless caching? 

The VM has certain sysctl and /proc tuning parameters. I have no
recollection of which of those flags exist in 2.4.18-3, so some
searching is probably in order. 

> Does the kernel have any interface to let you open a
> file for reading without it being cached?

a combination of mmap and madvise might do the job, or maybe
O_DIRECT. 

> I seem to remember something about
> O_DIRECT, but I don't know if it's relevant. Alternatively, is there a way
> to limit the amount of memory the kernel will use for disk caching?

See above, tuning parameters. 

Anyway, testing your workload on latest 2.4 / latest 2.5 and reporting
to lkml is exactly the kind of user reports the kernel developers
need. If you're up to compiling and booting a few times, consider
it... 
-- 
Muli Ben-Yehuda                                 http://www.mulix.org/   
mulix-BzGcCpaT2IbYtjvyW6yDsg@xxxxxxxxxxxxxxxx:~$ sctrace strace /bin/foo        
http://syscalltrack.sf.net/
Quis custodes ipsos custodiet?

pgpXDDeBqbZKu.pgp
Description: PGP signature