Relay-Version: version B 2.10 5/3/83; site utzoo.UUCP Path: utzoo!mnetor!seismo!rochester!ur-tut!tuba From: tuba@ur-tut.UUCP (Jon Krueger) Newsgroups: net.arch Subject: Re: Very large memories Message-ID: <683@ur-tut.UUCP> Date: Sat, 13-Sep-86 18:26:46 EDT Article-I.D.: ur-tut.683 Posted: Sat Sep 13 18:26:46 1986 Date-Received: Sat, 13-Sep-86 21:32:43 EDT References: <1164@ncr-sd.UUCP> <2383@peora.UUCP> Organization: Univ. of Rochester Computing Center Lines: 161 Summary: analysis of Joel's objections to virtual memory short description of VMS virtual memory comparison of virtual memory schemes In article <2383@peora.UUCP> joel@peora.UUCP (Joel Upchurch) writes: > . . . it seems to me that you'll use up [a particular performance > advantage of virtual memory] and a lot more in page faults. Since you > are reading the program piecemeal into virtual memory you are going to > be a lot slower because of the extra seek and rotational delays. It is > sort of like a guy driving on a surface street, instead of going over > and getting on the freeway. You get started faster, but you hit all > those traffic lights. The only case that is valid is if the > overwhelming majority of the pages in the program are never referenced. In terms of your analogy, if you are driving from A to B, it sure is faster to take the freeway. If you know in advance that you'll always want to start at A, stop at B, and hit all points in between, you neither need nor want local roads. But what if make trips where you decide to make additional stops along the way? What if you encounter conditions along the way that make continuing to B unnecessary? For instance, you discover that the foobars you usually pickup at B are available halfway to B. Or you hear the stock report on your car radio tell you that the foobar market has just collapsed. Better still, what if we gave you a road system that let you skip roads that don't contain any of your destinations? Then you'd find it faster to take local roads. Frequently you'd never get to B. What you would need is fast access to roads containing or adjacent to destinations and decision points. All the studies on typical memory referencing tell us that code behaves this way, and needs this kind of support to behave quickly. > Another consideration, is that on at least some virtual > memory systems I've worked with, the the system transfers > the program over to some sort of paging file, before it > starts, which means that you end up reading the whole > program in anyway, and writing it out to boot. Does anyone > know of a virtual memory system that doesn't work this > way? VMS. I'm not familiar enough with vmunix schemes, but I suspect they provide similar provisions. How VMS behaves: your very first instruction is started with no part of your code or data resident. Naturally this causes a page fault. VMS faults in your first cluster of pages from disk to physical memory. Your instruction is restarted and your image stays computable until a reference to another non-resident address causes another page fault. Your image continues executing indefinitely until (1) terminating (normally, interrupted by user, or fatal error condition forces image exit), (2) referencing another non-resident address causes another page fault, which again briefly interrupts execution, or (3) misc events like i/o or hibernation/suspension occur that pause image execution. Now. Your image can grow through continued faulting until it eats all available physical memory. So VMS establishes limits for each process called physical memory quotas. Your image grows up to its process's limit. Then, when handling the first fault that finds your image at its limit, VMS gives your image its next cluster, but in return, VMS takes away a cluster of your image's least-recently-used resident pages. If your image modified them ("dirty pages"), yes, VMS writes them to a paging file. But note the difference from the paging scheme you describe: the whole image is NOT written to a paging file before starting execution. In fact, images commonly run and terminate without exceeding their physical quotas. Images that do commonly find that their least-recently-used pages were never written to (initialization code, typically). So the paging file is there, but its function is to transparently handle a few cases. Two final clevernesses: you might think that if your image needs its modified page back from the paging file, another disk access is required. VMS keeps track of whether another process used your dirty page between the time you lost it and the time you needed it again. If not, VMS maps the physical page back into your space, no disk read required, not even a page copy in memory, cost of writing a few bytes of page table. Finally, VMS avoids writing to the paging file until all processes have created a total of more dirty pages than a system-wide limit. So if other process's demand for memory is low, VMS doesn't write to the paging file until your image grows to the total of your process's physical memory quota and the system's dirty page limit. > I'm no big fan of virtual memory. It made sense back in the > days of timesharing systems with expensive core memories, > but now with cheap semiconductor memory RAM disks and big > disk caches make more sense. Virtual memory is a series of tradeoffs best described as "getting more out of less". There will continue to be a need to do this indefinitely. Yes, memory gets cheaper, but programs get larger even faster. The demands of applications that benefit from or actually require more space will outpace the semiconductor industry's ability to provide physical memory at a given cost. It will always be convenient or necessary for some applications on some hardware to permit code to execute in spaces larger than current or available physical memory. For instance, my concept of a high-performance workstation involves a lot of processes executing on my behalf on my own local processor. The above scheme is said to let them share an arbitrary sized pool of local physical memory with low memory contention overhead, high flexibility to meet changing memory usage over time, good equity, and few pathological cases. So I think I want a similar scheme on a single-user machine with large, cheap, memory. To return to your analogy, if you want a fast way to get from A to B you take the highway. If cost is no factor, it would be faster yet to get those other cars off the road! They're just in your way, and require endless delays for safety rules that are only required to permit multiple cars safe use of a shared resource, the highway system. Now think how limited the performance is of driving your individual car around. You can only go one place at a time. While you're loading or unloading the car you're not actually getting anywhere. When you do have to make a long haul trip you're out of town for days at a time and miss opportunities otherewise available in a short trip. Your car is only so versatile, and can't be great for moving people and also for luggage. Wouldn't it be better, to get high performance, to buy a couple of cars, or a couple dozen, if their performance doubles and price halves every year, and their drivers, and keep them moving all over the interstate system on your behalf? Sure it would, but we'd all run out of major, minor, shared, and private roads in short order. Virtual memory, in this analogy, is getting by on the amount of roads we can afford this year by paving only the roads in use. > I worked at one large IBM mainframe site that installed > a external RAM disk drive on their MVS system. Do you > know what they used the RAM disk for? Page datasets! > In effect they were using RAM to simulate RAM! And the > worse part was it made sense since the IBM machine could > only use 16MB of real memory. It sounds like the hardware support and evolved system software environment provided spaces too small for applications. So it doesn't matter whether the spaces were physical or virtual, they were too small. So they resorted to kludges just as the one you mention (in fact, the same was done at the U of R. Maybe it still is). This kludge is pretty ugly, since physical memory will go wasted both in primary memory and RAM disks, since neither "sees" the other except through the processor. The processor thinks it has 16MB of memory and maybe another 16MB of fast disk. Paging on behalf of processes larger than 16MB will consist of page copies between the two. This is cheaper than disk speed transfers, but expensive compared to byte stores into page tables. Especially since the byte stores won't happen until really necessary, based on runtime conditions of image memory behavior and system-wide process memory demand. In summary, Joel, I think you've seen some inferior virtual memory schemes, and the entire concept of virtual memory has gotten a bad name with you. Good schemes cost less and deliver more. They solve problems now, and we'll continue to need them to solve similar problems in the future. They will be the same kinds of "getting more out of less" problems. But they'll reappear in different forms. This posting got pretty long, so some disclaimers: if I've exaggerated or denigrated or misrepresented VMS, DEC's not responsible. For extending Joel's analogy and the shakiness of my analogies, I alone am responsible. In case all net.arch poeple knew all the stuff I presented, I guess I've wasted your time and I apologize. To people who hate VMS, I apologize for liking some aspects of it. Anyway, I suspect that the good things that DEC coded into VMS in BLISS and MACRO would work just fine in 4.n BSD in C. -- --> Jon Krueger uucp: {seismo, allegra, decvax, cmcl2, topaz, harvard}!rochester!ur-tut!tuba Phone: (716) 275-2811 work, 235-1495 home BITNET: TUBA@UORDBV USMAIL: Taylor Hall, University of Rochester, Rochester NY 14627