Path: utzoo!news-server.csri.toronto.edu!cs.utexas.edu!sun-barr!olivea!uunet!mcsun!ukc!dcl-cs!aber-cs!athene!pcg From: pcg@cs.aber.ac.uk (Piercarlo Antonio Grandi) Newsgroups: comp.lang.lisp Subject: Re: Memory Management in Lisp? Message-ID: Date: 4 Mar 91 21:15:33 GMT References: <1991Feb15.191259.20090@aero.org> <1991Feb15.223520.17267@Think.COM> <1991Feb18.191549.7575@aero.org> <1991Feb19.030719.1137@Think.COM> Sender: pcg@aber-cs.UUCP Organization: Coleg Prifysgol Cymru Lines: 110 Nntp-Posting-Host: aberdb In-reply-to: jinx@zurich.ai.mit.edu's message of 27 Feb 91 02:46:34 GMT [ apologies if this appear more than once; previous attempts at posting may have not been successful because of a full partition ] I had written: pcg> To rely on garbage collection, which is a worst case mechanism to use to pcg> reclaim storage when nothing is known a priori about value lifetime, for pcg> known lifetime cases is clearly wasteful. An analogy is using dynamic pcg> type dispatching when the type of a value is well known by design. Maybe pcg> we need sophisticated, checkable lifetime declarations (better than pcg> DYNAMIC-EXTENT), not just sophisticated, checkable, type declarations. On 27 Feb 91 02:46:34 GMT, jinx@zurich.ai.mit.edu (Guillermo J. Rozas) commented: jinx> Hmm. So you would argue for programming with overlays rather than jinx> using a virtual memory system? Not at all, a well designed VM system is more flexible than overlays. But surely I think that over reliance on the default demand policies of a VM subsystem is catastrophic for performance, as amply demonstrated by any major application you can find around nowadays. When SunOS 3 became SunOS 4 a colossal performance hit was experienced because the default demand loading policy of VM is simply inappropriate to file access. GNU Emacs is another major and catastrophic example. jinx> Or having programmers think about disk sectors, head motion, and jinx> queueing operations on the disk rather than using the OS-supplied jinx> facilities that hide such details. Yes, most definitely yes. As long as the latency or bandwidth of disk and RAM are separated by some orders of magnitude most programmers have to carefully structure their programs around this simple fact. I have this idea that a very large, very large part of Computer Science is simply the consequence of dealing with this gap. Certainly most of OS and DBMS design is centered around it. Minimizing IO operations and memory usage is a large and important worry of any programmer's work. The OS-supplied facilities hide the ugly details of dealing with such facts; but the unpleasant facts remain, that application growth keeps pace with memory growth, and that disks are so slow, and bite hard. jinx> Of course, there are applications where performance is so critical jinx> that you cannot use virtual memory, or ignore details of disk jinx> access, etc., but the vast majority can. Here we are really on different planets. You sound like a vendor, or an agent of the Imperial MITI DRAM Service. Most nearly any "serious" application cannot be wholly memory resident and cannot be run with frequent collections; most any serious application writer has to be damn careful about locality of reference, and has to be damn careful about garbage generation rates. Relying on default replacement and reclamation policies simply doesn't cut it. The definition of "serious" is more or less the same as "mainstream". If one is unwilling to let Lisp/Scheme out in the mainstream then everything is simpler. But when you start considering for example as languages in which to do databases, numerical applications, compilers, operating systems, text processors and the like, caring about locality of reference and garbage generation rates pays off a lot. In short, do we want for Lisp/Scheme to remain a ghetto language for doing (even large scale!) AI demos or do we want to use Lisp/Scheme for programming ? :-) If the latter is desired, engineering considerations become important. As to me, I have been considering writing an OS kernel in Scheme, and so on. Current Scheme implementations are in the Lisp mould, unfortunately, but I can easily imagine Scheme being usable for all the above applications (something that is hard to imagine, even if it is feasible, for Common Lisp), even on a PC/AT class machine. jinx> Programmer productivity is greatly enhanced and the likelyhood of jinx> bugs is greatly decreased by using the system-supplied facilities, jinx> albeit at a cost in performance. I remember the old saw that says "an engineer can do for a dime what any fool can do for a dollar". If you have no resource constraints, more power to less prepared people, or more power to people that are more prepared in other fields. Admittedly as long as one has to write programs that are a few thousand lines long and run on immense personal workstations, there are no effective resource constraints. GNU Emacs is popular because its users all have 4-8MB each in their workstations. XyWrite is another story... In another newsgroup there was a remark that the exceptional success of Xenix/286 (yes, the 16 bit version) is due to the fact that a lot of VARs and their customers have discovered that a $1,000 286 gizmo can support 12 terminals doing business applications simply because it is the leanest and most efficient of its breed. Orders of magnitude away from Lisp, but amusing to note nonetheless. Nowadays that Lisp images are dozens of MBytes long and that people aspire to deliver them as products running on inexpensive machines, things are not longer that simple. Some Lisp programmers have had, much to their annoyance, to start reading books about IO operation minimizations written by database people. Garbage generations consideration have had to be discovered again. A promising start has been made towards putting Lisp programs out of their workspace ghetto and making them modular standalone applications that can interact with things like databases and other tools. -- Piercarlo Grandi | ARPA: pcg%uk.ac.aber@nsfnet-relay.ac.uk Dept of CS, UCW Aberystwyth | UUCP: ...!mcsun!ukc!aber-cs!pcg Penglais, Aberystwyth SY23 3BZ, UK | INET: pcg@aber.ac.uk