Relay-Version: version B 2.10 5/3/83; site utzoo.UUCP
Path: utzoo!mnetor!seismo!cmcl2!lanl!jlg
From: jlg@lanl.ARPA (Jim Giles)
Newsgroups: net.arch
Subject: Re: VERY LARGE main memories
Message-ID: <7331@lanl.ARPA>
Date: Wed, 10-Sep-86 16:44:49 EDT
Article-I.D.: lanl.7331
Posted: Wed Sep 10 16:44:49 1986
Date-Received: Wed, 10-Sep-86 20:18:40 EDT
References: <1130@bu-cs.bu-cs.BU.EDU> <7144@lanl.ARPA> <7148@lanl.ARPA> <7094@utzoo.UUCP>
Reply-To: jlg@a.UUCP (Jim Giles)
Organization: Los Alamos National Laboratory
Lines: 39

In article <7094@utzoo.UUCP> henry@utzoo.UUCP (Henry Spencer) writes:
>...
>With some reason.  What you're saying is that because the operating-system
>people are too lazy to devise paging algorithms that are useful for large
>scientific programs, the programmers should be required to do it themselves.
>Apart from the matter of constantly reinventing the wheel, there is also
>the problem that it's a lot of work to get it right -- program reference
>patterns are notorious for being hard to predict beforehand, which means
>experimenting and then twiddling the code to match the results.

It's not just that the operating system or hardware designers are too lazy
to come up with a good scheme.  The problem is that any scheme they DO come
up with must work for general cases.  That is, it can't take advantage of
special knowledge of a specific algorithm.

The individual applications programmer CAN take advantage of such knowledge.
To be sure, this is an expensive and difficult programming project.  But,
if you've just spent $10-$20 million on a fast machine, you aren't going
to balk at a few million more in programmer man-hours to get the speed
that you shelled out so much cash for.

Since hardware (for whatever reason) can run faster without virtual memory,
there will always be a market among the high-end users for machines that
don't have it.  Since these are the sort of machines with the Very large
memories that are talking about, I question the desirability of virtual
memory on them.

As a final note: there are some types of algorithm for which it is
extremely easy to predict the data usage patterns.  Most Finite-Difference
and Finite-Element codes are of this kind.  These applications involve a
small number of very large arrays which are referenced cyclically.  Other
applications, like image manipulation and particle transport, have somewhat
more difficult patterns, but there are known methods for dealing with them.
These are the types of codes which form the predominant workload for
today's large memory supercomputers (whether bought by oil companies or
government).  Now, it might be convenient to implement virtual memory
schemes which are useful in this context, but I doubt that the extra
overhead in the memory interface would be justified - especially since
the explicit methods for dealing with them are fairly easy to implement.