Xref: utzoo comp.arch:11100 comp.sys.mips:107
Path: utzoo!attcan!uunet!ginosko!gem.mps.ohio-state.edu!csd4.csd.uwm.edu!mailrus!cornell!uw-beaver!ubc-cs!alberta!calgary!ctycal!ingoldsb
From: ingoldsb@ctycal.COM (Terry Ingoldsby)
Newsgroups: comp.arch,comp.sys.mips
Subject: Re: Memory utilization & inter-process contention
Summary: Response time vs throughput
Message-ID: <440@ctycal.UUCP>
Date: 23 Aug 89 18:38:11 GMT
References: <3332@blake.acs.washington.edu> <1989Aug22.163100.25540@utzoo.uucp> <3342@blake.acs.washington.edu>
Organization: The City of Calgary, Ab
Lines: 47

In article <3342@blake.acs.washington.edu>, lgy@blake.acs.washington.edu (Laurence Yaffe) writes:
> In article <1989Aug22.163100.25540@utzoo.uucp> henry@utzoo.uucp (Henry Spencer) writes:
> >In article <3332@blake.acs.washington.edu> lgy@newton.phys.washington.edu (Laurence Yaffe) writes:
> >>suppose you have a 32 Mbyte system and that there are only two processes
> >>running, each of which uses 25 Mbytes of virtual memory (almost all data space)
> >>with rapid, random access patterns...
> >>    This sort of contention could obviously be prevented by a smarter
> >>scheduling technique - something which would entirely swap out one
> >>process for a period of time in order to let the other process run.
> >
> >Have you ever figured out how long it takes to swap out a 25MB process?
> >Or to swap it back in again?  The fact is, this example system simply
> >does not have enough main memory for its workload.  The fix is, either
> >double the memory or run the jobs in sequence rather than simultaneously.
> >
> >V7 /bin/mail source: 554 lines.|     Henry Spencer at U of Toronto Zoology
> >1989 X.400 specs: 2200+ pages. | uunet!attcan!utzoo!henry henry@zoo.toronto.edu
> 
>     This response ("buy more memory, or sequence jobs by hand"), while common,
> completely misses the point.  The real issue here is how to maximize overall

But surely this is a discussion of the classis tradeoff of response time vs
throughput.  It is quite common that tuning a system for maximum performance
decreases overall response time to the point that users threaten to lynch the
system manager.  Note that most supercomputers, whose raison d'etre is to go
real fast, don't timeshare at all.

Finally, I don't understand why your performance is so poor, but maybe there
is something I don't understand about UNIX scheduling, or else your task is
*so* degenerate that no memory management/scheduling scheme will work.  In
normal circumstances if a process tries to access a non-resident page, the OS
blocks that process until the I/O completes and lets the other job(s) that
do have the necessary info in memory continue to run.  If you mean that your
jobs are so degenerate that each process does only a handful of memory accesses
before page faulting, then obviously you will always be waiting for the
page to be swapped in.  If your accesses are truly random, then *no* memory
scheme will ever be able to give you a good hit rate.  If they are not
truly random, then organize your data such that elements are stored in
such a way that chronologically accessed data reside on the same page.

In summary, either reorganize your data, or do what Henry says.
Although interesting, this discussion is probably moving away from the
charter or comp.arch.
-- 
  Terry Ingoldsby                       ctycal!ingoldsb@calgary.UUCP
  Land Information Systems                           or
  The City of Calgary         ...{alberta,ubc-cs,utai}!calgary!ctycal!ingoldsb