Relay-Version: version B 2.10 5/3/83; site utzoo.UUCP Path: utzoo!mnetor!seismo!rutgers!ucla-cs!ames!ucbcad!ucbvax!YALE.ARPA!LEICHTER-JERRY From: LEICHTER-JERRY@YALE.ARPA Newsgroups: comp.os.vms Subject: Re: Virtually nuked... Message-ID: <8706230916.AA02373@ucbvax.Berkeley.EDU> Date: Tue, 23-Jun-87 05:17:21 EDT Article-I.D.: ucbvax.8706230916.AA02373 Posted: Tue Jun 23 05:17:21 1987 Date-Received: Wed, 24-Jun-87 07:07:07 EDT Sender: daemon@ucbvax.BERKELEY.EDU Reply-To: Distribution: world Organization: The ARPA Internet Lines: 84 [The message I'm responding to asked for help with FORTRAN programs containing very large arrays, which caused the Linker to run out of virtual memory.] There is only one ultimate cure for not having enough virtual memory: In- crease VIRTUALPAGECNT! You report that you are using a value of around 46,000. On the 8600 I'm writing this from, VIRTUALPAGECNT is 250,000. The system has 16 Meg of real memory - not very large by modern standards, actu- ally. It works just fine. The value was set that high because we, too, have run some large scientific programs - flame simulations, that kind of thing. Actually, I suspect the largest virtual size belonged to a program that did neural net simulations. The main thing VIRTUALPAGECNT will cost you is REAL memory - though not very much: Every 128 pages requires 4 bytes of permanently-reserved system memory per balance set slot; the number of slots is set by BALSETCNT. So this sys- tem, which has BALSETCNT=100, is reserving (250000/128)*4*100 bytes (781,600 after rounding), or 1527 pages out of 32768. Not really significant. Don't be mislead by the above calculation into thinking that VIRTUALPAGECNT is free. There are hidden costs: a) While you can have a huge virtual image running in a very small working set, the result isn't going to make anyone happy - thrashing is a good way to exercise disks, but that's about all that can be said for it. This system has WSMAX set to 16000. Most people's UAF entries specify much lower values - 1024 or 2048 is typical. Those people who are running large images are, of course, given larger limits. (Avoid giving out VERY large WSQUOTA values; instead, give people a large WSQUOTA - say, 4000 - and a very large WSEXTENT. Then encourage them to run overnight. You could even set up batch queues with larger working set quotas, so that the real biggies can run ONLY when they won't screw everyone else up.) Implicit in this recommendation, of course, is the requirement that you actually have the physical memory. There's no get- ting away from this. As a friend (Martin Minow) has com- mented: Virtual memory is fine if you want to do virtual work. Enlarging on this just a bit: With the array sizes you dis- cussed, it's not going to be possible to fit everything into the working set at once anyway. Dealing with very large arrays in a paging environment takes some care, as fairly trivial changes - interchanging the order of a pair of loops driving the subscripts of a 2D array, for example - may cause a two order of magnitude difference in performance. These problems have been well studied in the scientific programming community; if your users are not familiar with this kind of work, make sure they talk to someone who is. (There are a number of people here who I'm sure would be willing to con- sult....) b) You'll also need large page files to map those huge FORTRAN arrays your people need - they have to live SOMEWHERE when they aren't in main memory. This 8600 has a little over 100,000 blocks of pagefile. That's NOT enough for the real monsters. The neural net simulation I mentioned, for example, needed more space than that. There is a way around the problem, which we took, but it does require some additional work: Rather than building the arrays into the program, create them as disk file sections using SYS$CRMPSC. The advantage of this is that you can create the disk file on any disk where you have free space, even on a temporary basis; you don't have to tie space up on a long-term basis. Also, you can charge the particular users who need the large disk resources directly, rather than having tens of thousands of blocks of space go into a page file that you can't charge anyone for, and which no one uses most of the time. For the user, a disk file section has another advantage: You can save the file, thus getting an instant checkpoint of the array at some point in the computation. Much faster and sim- pler than writing the whole array out with I/O statements, then reading it back later. The details are too elaborate to go into here; see the System Services book for details on disk file sections. Unfortunat- ely, I don't know of any references on using them for the kind of thing you need. -- Jerry -------