Path: utzoo!utgpu!jarvis.csri.toronto.edu!clyde.concordia.ca!uunet!tut.cis.ohio-state.edu!ucbvax!imf.unit.no!hanche From: hanche@imf.unit.no (Harald Hanche-Olsen) Newsgroups: comp.sys.apollo Subject: PRISM Fortran compiler causes trashing! Message-ID: Date: 10 Dec 89 15:02:59 GMT Sender: daemon@ucbvax.BERKELEY.EDU Organization: The Internet Lines: 49 This is a short tale about a long FORTRAN compilation on the DN10000. I was to compile a FORTRAN program with a total of 120K lines split up into 37 files. Since our machine has only 8MB of memory and compiling is known to load it down severely, I set it up to start compiling at 1am. Next morning I got in to the office sort of late and found that the compilation was still running, and students were complaining loudly about the sluggishness of the machine. So I killed it, and restarted it the following night. History repeated itself next morning, which got me to thinking. By now, I had compiled 87K lines, which had taken 29.5 hours of elapsed time and a total of 5 million page faults! "All right", I thought, "maybe the compiler tries to compile all subroutines in one file simultaneously, thereby filling up a lot of memory and causing page trashing. In other words, maybe it helps to split the files into single subroutine pieces?" To test my hypothesis, I took a representative FORTRAN file of 2850 lines and split it up into little files using `fsplit'. I got 19 files with sizes ranging from 22 lines to 280 lines. Compiling them all and timing with the `time' command, I got the answer 289.3u 20.4s 14:51 34% 0+0k 1014+0io 30083pf+0w For those who don't know `time' output format, this means 289 seconds user time, 20 seconds system time (i.e., spent in the kernel), almost 15 minutes elapsed time, and 30000 page faults. Compare this with the result of trying to compile the whole file in one shot. After tying up the machine for over two hours, I decided the most merciful thing to do was to kill it, yielding this result: 267.2u 271.5s 2:17:10 6% 0+0k 7043+0io 441500pf+0w That is, 9 times as much elapsed time, 14 times as many page faults! Needless to say, I decided to split up the rest of the input files and compile them all separately. Thus, the remaining 32K lines took a total of 2 hours elapsed time and 300000 page faults. That is 266 lines per (elapsed) minute, compared to 67 lines/minute for the first part of the compilation! Not quite the same performance ratio as in the test case, but still quite a difference. Technical detail: We are running under SR10.1.p. Our Fortran compiler is version 10.5.p. As I said, we have 8M of memory. (We will upgrade to 32M soon). Conclusion: Split your files before compiling! Rhetoric question: What do those compiler writers at HP/apollo think they're doing? Writing for machines with a minimum of 128M of memory? - Harald Hanche-Olsen Division of Mathematical Sciences hanche@imf.unit.no The Norwegian Institute of Technology hanche@norunit.bitnet N-7034 Trondheim-NTH NORWAY