Path: utzoo!utgpu!jarvis.csri.toronto.edu!mailrus!tut.cis.ohio-state.edu!brutus.cs.uiuc.edu!apple!vsi1!wyse!mips!mash
From: mash@mips.COM (John Mashey)
Newsgroups: comp.arch
Subject: Re: Memory utilization & inter-process contention
Message-ID: <26504@winchester.mips.COM>
Date: 30 Aug 89 02:31:30 GMT
References: <3332@blake.acs.washington.edu> <261500008@S34.Prime.COM> <249@gp.govt.nz> <2389@auspex.auspex.com> <4176@drilex.UUCP>
Reply-To: mash@mips.COM (John Mashey)
Organization: MIPS Computer Systems, Inc.
Lines: 72

In article <4176@drilex.UUCP> dricejb@drilex.UUCP (Craig Jackson drilex1) writes:
......
>I think that the VMS guys thought that the ability to recover pages off
>the free list gave them many of the features of reference bits, without
>either the memory traffic to update them or the software effort to read
>them.
>
>P.S. The largest processors of the Unisys A-Series still don't have
>modified bits.  The designers decided that the memory traffic would be
>too high.  However, code and read-only arrays (declared at compile time)
>has been marked specially for years, and need not be paged out.

Say some more.  I'd assume there's a trap-on-write bit or equivalent
somewhere, but no modified bit set by hardware?

Note that on a paged machine running UNIX, with typically-sized pages
(i.e., fairly different from A-series, I think), hardware-set modified
bits are unlikely to consume much bandwidth, unless done by blindly
writing a page table entry upon every write (as opposed to state
changes). This why it is easy to get away with trapping writes to
set modify bits instead of having the hardware do it.

In UNIX, the frequency of the state change:
	writable, but not yet modified -> modified
is:
	a) far less than references, TLBmisses, or similar actions.
	b) Occurs in the following cases:
		1) Write to a page that is demand-fill-zero (like a
			stack section).
		2) Read to a demand-fill-zero page, later followed by
			a write.
		3) Write to a demand-load page (.data).
		4) Read a demand-load page, later followed by a write.
		5) Write to a copy-on-write page inherited from fork,
			or any other reason to use copy-on-write.

Now, In cases 1) and 3), you know you were doing a write, so you can
mark the page modified in the first place, as soon as you've zeroed or
loaded it.  In case 2), you might as well do that also, as reasonable
code sequences are unlikely to leave a .bss or sbrk'd hunk of memory
clean for very long.  In case 5), everybody turns off the ability of
hardware to do a write without taking an interrupt, anyway, because the
OS has significant work to do.
That leaves 4), where you reference some .data, but go for a longtime
without doing a write to the page.  (Note that this is more likely than
the same sequence for a DFZ-page, since you might have large tables of
data that are effectively read-only, but could be modified.)
item 4) is clearly bounded by the rate of demand-load pageins/second of
data (not code), which varies from system to system, but is usually
much less than the number of disk I/Os.  As an example, I took a quick
look at one of our M/2000s, which had 300 processes, and was doing 500
context-switches a second (I have no idea what it was doing). A bound on the
number for 4) is the pgfil number from vsar (plus some of the swpin).
These numbers were usually = 0 (this is a big machine), but I did see
them get as high as 5/second after watching for 5 minutes.  Thus,
hardware-set modify bits would save you about 5 exceptions/second.
On a smaller machine, the number might be higher, of course.
The machine was doing about 1000 syscalls/sec, for context.

BOTTOM LINE: either you want to trap a write and do something complicated
anyway, in which case you turn off any hardware's ability to set a
modified bit, or you're saving at most one exception per page of
paged/swapped-in data.

I can't dig up a reference, but I'd swear I've seen at least once,
a CPU which had at least one bug in setting modify bits somewhere,
i.e., the hardware to do this is not always trivial....
-- 
-john mashey	DISCLAIMER: <generic disclaimer, I speak for me only, etc>
UUCP: 	{ames,decwrl,prls,pyramid}!mips!mash  OR  mash@mips.com
DDD:  	408-991-0253 or 408-720-1700, x253
USPS: 	MIPS Computer Systems, 930 E. Arques, Sunnyvale, CA 94086