Mozy Online Backup: 2GB Free. Automatic. Secure.

Subject: Re: [PATCH] 2.6.4-rc2-mm1: vm-split-active-lists - msg#00056

List: linux.kernel.mm

Date: Prev Next Index Thread: Prev Next Index

Mark_H_Johnson@xxxxxxxxxxxx wrote:

Nick Piggin <piggin@xxxxxxxxxxxxxxx> wrote:

Andrew Morton wrote:

That effect is to cause the whole world to be swapped out when people
return to their machines in the morning. Once they're swapped back in

the

first thing they do it send bitchy emails to you know who.

From a performance perspective it's the right thing to do, but nobody

likes

it.

Yeah. I wonder if there is a way to be smarter about dropping these
used once pages without putting pressure on more permanent pages...
I guess all heuristics will fall down somewhere or other.

Just a question, but I remember from VMS a long time ago that
as part of the working set limits, the "free list" was used to keep
pages that could be freely used but could be put back into the working
set quite easily (a "fast" page fault). Could you keep track of the
swapped pages in a similar manner so you don't have to go to disk to
get these pages [or is this already being done]? You would pull them
back from the free list and avoid the disk I/O in the morning.

Not too sure what you mean. If we've swapped out the pages, it is
because we need the memory for something else. So no.

One thing you could do is re read swapped pages when you have
plenty of free memory and the disks are idle.

By the way - with 2.4.24 I see a similar behavior anyway [slow to get
going in the morning]. I believe it is due to our nightly backup walking
through the disks. If you could FIX the retention of sequentially read
disk blocks from the various caches - that would help a lot more in
my mind.

updatedb really wants to be able to provide better hints to the VM
that it is never going to use these pages again. I hate to cater for
the worst possible case that only happens because everyone has it as
a 2am cron job.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@xxxxxxxxxx For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"aart@xxxxxxxxx";> aart@xxxxxxxxx </a>

Was this page helpful?

Thread at a glance:

Previous Message by Date: click to view message preview

Re: [PATCH] 2.6.4-rc2-mm1: vm-split-active-lists

Nick Piggin <piggin@xxxxxxxxxxxxxxx> wrote: >Andrew Morton wrote: >>That effect is to cause the whole world to be swapped out when people >>return to their machines in the morning. Once they're swapped back in the >>first thing they do it send bitchy emails to you know who. >> >>>From a performance perspective it's the right thing to do, but nobody likes >>it. >> >> > >Yeah. I wonder if there is a way to be smarter about dropping these >used once pages without putting pressure on more permanent pages... >I guess all heuristics will fall down somewhere or other. Just a question, but I remember from VMS a long time ago that as part of the working set limits, the "free list" was used to keep pages that could be freely used but could be put back into the working set quite easily (a "fast" page fault). Could you keep track of the swapped pages in a similar manner so you don't have to go to disk to get these pages [or is this already being done]? You would pull them back from the free list and avoid the disk I/O in the morning. By the way - with 2.4.24 I see a similar behavior anyway [slow to get going in the morning]. I believe it is due to our nightly backup walking through the disks. If you could FIX the retention of sequentially read disk blocks from the various caches - that would help a lot more in my mind. --Mark H Johnson <mailto:Mark_H_Johnson@xxxxxxxxxxxx> -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@xxxxxxxxxx For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"aart@xxxxxxxxx"> aart@xxxxxxxxx </a>

Next Message by Date: click to view message preview

Re: [PATCH] 2.6.4-rc2-mm1: vm-split-active-lists

Nick Piggin <piggin@xxxxxxxxxxxxxxx> wrote: >Mark_H_Johnson@xxxxxxxxxxxx wrote: >>Nick Piggin <piggin@xxxxxxxxxxxxxxx> wrote: >> >>>Andrew Morton wrote: >>> >> >>>>That effect is to cause the whole world to be swapped out when people >>>>return to their machines in the morning. Once they're swapped back in >>>> [this is the symptom being reported] >>Just a question, but I remember from VMS a long time ago that >>as part of the working set limits, the "free list" was used to keep >>pages that could be freely used but could be put back into the working >>set quite easily (a "fast" page fault). Could you keep track of the >>swapped pages in a similar manner so you don't have to go to disk to >>get these pages [or is this already being done]? You would pull them >>back from the free list and avoid the disk I/O in the morning. > >Not too sure what you mean. If we've swapped out the pages, it is >because we need the memory for something else. So no. Actually - no, from what Andrew said, the system was not under memory pressure and did not need the memory for something else. The swapping occurred "just because". In that case, it would be better to keep track of where the pages came from (i.e., swap them in from the free list). Don't get me wrong - that behavior may be the "right thing" from an overall performance standpoint. A little extra disk I/O when the system is relatively idle may provide needed reserve (free pages) for when the system gets busy again. >One thing you could do is re read swapped pages when you have >plenty of free memory and the disks are idle. That may also be a good idea. However, if you keep a mapping between pages on the "free list" and those in the swap file / partition, you do not actually have to do the disk I/O to accomplish that. --Mark H Johnson <mailto:Mark_H_Johnson@xxxxxxxxxxxx> -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@xxxxxxxxxx For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"aart@xxxxxxxxx"> aart@xxxxxxxxx </a>

Previous Message by Thread: click to view message preview

Next Message by Thread: click to view message preview

Re: [PATCH] 2.6.4-rc2-mm1: vm-split-active-lists

Nick Piggin wrote: > One thing you could do is re read swapped pages when you have > plenty of free memory and the disks are idle. Better: re-read swapped pages _and_ file-backed pages that are likely to be used in future, when you have plenty of free memory and the disks are idle. updatedb would push plenty of memory out overnight. But after the cron jobs and before people wake up in the morning, the kernel would gradually re-read the pages corresponding to mapped regions in processes. Possibly with emphasis on some processes more than others. Possibly remembering some of that likelihood information even when a particular executable isn't currently running. During the day, after a big compile the kernel would gradually re-read pages for processes which are running on your desktop but which you're not actively using. The editor you were using during the compile will still be responsive because it wasn't swapped out. The Nautilus or Mozilla that you weren't using will appear responsive when you switch to it, because the kernel was re-reading their mapped pages after the compile, while you didn't notice because you were still using the editor. The intention is to avoid those long stalls where you switch to a Mozilla window and it takes 30 seconds to page in all those libraries randomly. It's not necessary to keep Mozilla in memory all the time, even when the memory is specifically useful for a compile, to provide that illusion of snappy response most of the time. -- Jamie -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@xxxxxxxxxx For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"aart@xxxxxxxxx"> aart@xxxxxxxxx </a>

Loading Comments...