Path: utzoo!censor!geac!torsqnt!news-server.csri.toronto.edu!mailrus!uwm.edu!zaphod.mps.ohio-state.edu!samsung!munnari.oz.au!sirius.ucs.adelaide.edu.au!cs.ua.oz.au!francis
From: francis@chook.ua.oz.au (Francis Vaughan)
Newsgroups: comp.arch
Subject: Re: Paging page tables & SUN4s
Message-ID: <1122@sirius.ucs.adelaide.edu.au>
Date: 6 Jul 90 13:30:48 GMT
References: <3300142@m.cs.uiuc.edu> <1990Jun29.154940.22762@tera.com> <2936@skye.ed.ac.uk> <1990Jul06.003628.2633@esegue.segue.boston.ma.us>
Sender: news@ucs.adelaide.edu.au
Reply-To: francis@chook.ua.oz.au (Francis Vaughan)
Lines: 54

In article <1990Jul06.003628.2633@esegue.segue.boston.ma.us>,
johnl@esegue.segue.boston.ma.us (John R. Levine) writes:
|>I don't get what this argument is about.  Every machine I've ever seen that
|>has regular (as opposed to inverted) page tables pages the page tables.
|>Some, like the Vax, do it explicitly by embedding the page table in another
|>pagable address space.  Most others, such as the IBM 370 and the Intel 386,
|>get exactly the same effect with two-level page tables.  In both cases, the
|>page table is broken up into chunks (which always seem to have a size of one
|>page) and each chunk can be marked resident or non-resident independently.

Umm, No. Try the SUN Sparc MMU. SUN-OS pages its copies of page maps (called
pmegs). However the hardware has its own local copy of a subset of the 
tables. How much is dependant on the machine. SUN-OS maintains the illusion
of pageable page tables but the underlying hardware is very different.

Each pmeg holds 32 entries (or on the SS-1 64 entries) where each entry
refers to a page. SS-1s have MMU memory for only 128 pmegs or 64*128 =8192
page entries. That means it can only hold entries for 32 megabytes of
virtual memory agragate for all processes running on the machine. Since 
that is in chunks of 64*4=256KB and you need a seperate chunk (therefore 
pmeg) for data, bss, code, no matter how little is used the  system rapidly
runs  out of pmegs.

If the OS needs a pmeg and the hardware table is full, it selects a used 
one and trashes it. This is done with a software cache manager that copies
the kernel version of the table into the hardware table. You take quite a 
perfomance hit when this happens. The crazy thing is that SUN-OS places all 
the pages refered to by the reused pmeg onto the free list. If a process 
gets all of its pgems reused then all of its pages are marked as free then
it gets marked for eager swapping by the kernel. This can result in 
processes that are actually entierly memory resident being swapped. An 
easy way to tell is if the free list is huge, but the system is thrashing.
                  
The hit for SS-1s (and this includes SS-1+s) is that you don't get the sort
of improvement you would expect in performance when adding much more than
about 16MB of memory, especially if there is little image sharing. Extra
memory is essentially used as a disk cache. 

The bigger machines (especially the 490) have bigger slabs of MMU memory 
and do not suffer as much (the 490 has 1024 pmegs of 32 entries each).

Currently I do not know if SUN intend fixing the problem in SUN-OS (this 
has been coloquially known as the pmeg leakage problem) however they have 
announced a new product that may help. The Sun Database Excelerator.

" This product features: IMPROVED MEMORY MANAGEMENT; ..."
" This easy to install unbundled software dramaticly increases Sun 
SPARCserver performance with database applications ..."
"...increased system throughput and improved response times particularly 
under  heavy system loads. Maximum throughput increases up to 50% while 
supporting up to five times as many concurrent users."


						Francis Vaughan