Path: utzoo!utgpu!jarvis.csri.toronto.edu!mailrus!bbn!rochester!udel!burdvax!barry From: barry@PRC.Unisys.COM (Barry Traylor) Newsgroups: comp.arch Subject: Re: Memory utilization & inter-process contention Summary: A Series workingset management defended Keywords: workingset management, A Series Message-ID: <11294@burdvax.PRC.Unisys.COM> Date: 30 Aug 89 05:43:49 GMT References: <3332@blake.acs.washington.edu> <261500008@S34.Prime.COM> <249@gp.govt.nz> <2389@auspex.auspex.com> <4176@drilex.UUCP> Followup-To: comp.arch Organization: Unisys Corporation, Paoli Research Center; Paoli, PA Lines: 81 In article <4176@drilex.UUCP> dricejb@drilex.UUCP (Craig Jackson drilex1) writes: >The older operating system, called MCP, on the Unisys A-Series did >classic working set maintenance, without either a referenced bit >or a modified bit. It did use some form of a clock algorithm to >figure out what to throw out, but maintained working sets to determine >how much of each task's memory to throw out. The working set calculations >were based on things like the number of segments faulted-in in a given >period and their size. While I'm glad you like the older algorithms, you fail to mention that workingsets were maintained by stepping linearly through memory and forcing data and code segments out to disk. While stats were kept for each process's memory usage, no intellegence was applied to the selection of segments to overlay. You also failed to note that on split code/data machines (B7900s and ASN A15s), code was not subjected to the workingset mechanism at all. In any rate, to come up with those nifty statistics, the entire memory space of the process was forced out of memory. >Interestingly, the operating system for their latest architecture, called >MCP/AS, punts all of the working set stuff, replacing it with a simple >clock algorithm with two hands. There is only one memory pool, consisting >of the entire machine. This allows a low-priority user task to suck up >lots of memory, until both it and high-priority tasks begin to page. The >low-priority task doesn't mind, but everybody else suffers. (Lots of the >the code which is equivalent of the Unix kernel is pagable on this system.) >The vendor response was to buy more memory; we're up to 144MB and still >hurting occasionally. Their next response was "Don't run big-memory jobs". First, I was solely responsible for the defective MCP/AS workingset mechanism, which was intended to be fool-proof. Unfortunately it spent most of its time fooling itself, alternating between doing too much and doing too little. The mechanism has been largely reimplemented (by me again, so send complaints to the same address) on the 38 release and is much more predictable and reliably tunable. You failed to mention that the MCP/AS mechanism is based on an LRU concept, and had it worked as I had expected (which, of course, you could not have known), it would have been much better in terms of overall throughput than the ASN (old) mechanism. My experimentation with the 38 version shows that it truly controls workingsets while minimally consuming system resources. Your milage will, of course, vary. Also, I'm not sure, but I think that SUSPENDER can still be run. SUSPENDER forces low priority tasks to stop executing if thrashing is detected (I think). >This issue points up one other thing: this operating system runs fine as >a transaction-processing monitor, where there are relatively few tasks >to be run, and each task performs a well-defined function. The MCP/AS release 38 workingset mechanism works well for any task that hangs around for more than a couple of minutes. Workingset control is largely only an issue on long running tasks that, over the course of the run, consume large quantities of memory, but at any given period of time use a relatively minor subset of that total. The workingset mechanism will remove from memory those segments that are not indicating any use WITHOUT FORCING THEM OUT TO DISK. Those segments in use will remain in memory unless DEMAND overlay activity kicks them out. Of course for the MCP/AS workingset mechanism, there are a couple of tunable parameters and the whole thing can be turned off, if desired. The fundamental philosophy behind the mechanism is that memory allocations are faster if there is available memory, and processes run just as well accessing the current workingset with just the workingset in memory as they do when all of the memory segments are resident. I'm not sure what you mean by "few tasks". Most of the time our in-house A17 shows more than 200 active tasks, not all of which are well behaved. >P.S. The largest processors of the Unisys A-Series still don't have >modified bits. The designers decided that the memory traffic would be >too high. However, code and read-only arrays (declared at compile time) >has been marked specially for years, and need not be paged out. The modified bit seemed like a good idea at the time. We have never been able to gather statistics about the effectiveness of it. Part of the problem is that the data must be written out to disk at least once and once written, all that's saved are subsequent writes IF the data is not modified. In the mean time the processor flows are fouled up on every memory write operation since the modified bit must be checked. On a highly concurrent processor such is used with the A15/A17, the extra state required to be kept with every segment reference seemed too high of a price to pay for an unknown (and still unproven) return. Barry Traylor (Yup, that's right, my opinions are my own, NOT UNISYS's) Unisys A Series Engineering (Tredyffrin) (barry@prc.unisys.com)