Path: utzoo!utgpu!attcan!uunet!lll-winken!ames!ncar!tank!uxc!uxc.cso.uiuc.edu!uxg.cso.uiuc.edu!uicbert.eecs.uic.edu!wilson
From: wilson@uicbert.eecs.uic.edu
Newsgroups: comp.lang.lisp
Subject: big programs?  data use?
Message-ID: <63200006@uicbert.eecs.uic.edu>
Date: 6 Jan 89 03:36:00 GMT
Lines: 45
Nf-ID: #N:uicbert.eecs.uic.edu:63200006:000:2109
Nf-From: uicbert.eecs.uic.edu!wilson    Jan  5 21:36:00 1989


I am trying to get a handle on overall memory usage by serious Lisp programs.

I used to thing that serious programs on big systems (e.g., Symbolics, TI
Explorer) often had a large amount of data that was live at any given time.
Now I am not so sure.

I am moving toward the view that most programs never use more than a few
megabytes of data, and that very few ever have even one megabyte of data
that is live at any given time.  (That is, even if they allocate a dozen
or so megabytes, over 90% of it dies very fast and there's really not
all that much around.)

Does anybody want to disabuse me of this impression?  Have your programs
ever had a megabyte survive a garbage collection?  (Without scavenging
system code and data, that is.)

Has anybody out there ever had a program with ten megabytes of live data?
Twenty?  (Including big arrays for numerical programs, but maybe not
counting pixmaps for graphics.)

If anybody has any real data on this, or reasonable anecdotal
reports, please respond.  If your gc gives statistics after a scavenge,
I'd be interested in the kind of numbers you see, both typically and
occasionally.  (Things like the size of older generations and amount
surviving, about how often they have to be collected, etc.).

I'm trying to figure out how much the memory requirements and paging costs
in Lisp systems is because of the programs they run, and how much is
accounted for by the system itself.  I would like to get a better handle
on what accounts for most locality problems:  system vs. user, code vs.
data, etc.

Note:  I'm not implying systems shouldn't use considerable resources if
       it's necessary to provide the desired functionality.  And I know
       that increased size of the system (e.g. more builtin functions)
       can decrease the apparent size of user code.  But knowing
       where the problems are is always good, even if they're there
       for good reasons.


Paul R. Wilson                         
Human-Computer Interaction Laboratory
U. of Illin. at C. EECS Dept. (M/C 154)   wilson%uicbert@uxc.cso.uiuc.edu
Box 4348   Chicago,IL 60680