Xref: utzoo comp.emacs:8999 gnu.emacs:3741
Path: utzoo!utgpu!watserv1!watmath!uunet!cs.utexas.edu!yale!mintaka!spdcc!bu.edu!nntp-read!jbw
From: jbw@bucsf.bu.edu (Joe Wells)
Newsgroups: comp.emacs,gnu.emacs
Subject: Re: GNU EMACS: non-buffer memory usage
Message-ID: <JBW.90Sep2200647@bucsf.bu.edu>
Date: 3 Sep 90 00:06:47 GMT
References: <1537@anaxagoras.ils.nwu.edu>
Sender: news@bu.edu.bu.edu
Followup-To: comp.emacs
Organization: Boston University Computer Science Department
Lines: 57
In-reply-to: krulwich@ils.nwu.edu's message of 29 Aug 90 18:25:01 GMT

In article <1537@anaxagoras.ils.nwu.edu> krulwich@ils.nwu.edu (Bruce Krulwich) writes:

   Ignoring all the issues of user interface, I'm concerned about GNU
   EMACS's ability to react well to reasonably large amounts of
   information stored in CONS-cells and strings as opposed to editor
   buffers.  My question is:

   Is the EMACS-LISP heap from which CONS cells and strings are given
   memory handled differently than the memory in which editor buffers are
   stored, and is the former worse for storing information than the
   latter?

With GNU Emacs 18, memory for both types of storage is allocated using
malloc.  Thus, they both use the same "type" of memory.  Using a buffer
for storing a large database may allow much better virtual memory
performance if you access the buffer in a non-expensive manner.  Expensive
types of access to a buffer include lots of random-access modifications
and, to a lesser degree, any other random access to the buffer.

Buffer modifications can be expensive because any modification requires
moving the "gap" in the buffer to the point of modification.  If you make
a change at the beginning of the buffer, and then one at the end, every
byte in the buffer will be moved (probably from one page of memory to the
next).  Normal editing usually doesn't require moving the gap very far or
very often.  Thus, buffer operations are very cheap for editing.

Of course there are also some obvious reasons for not using lists either.
Using lists, it is possible for each cons cell to be located in a
different virtual memory page.  If you are accessing a large list, you
will quickly exhaust your active set and start forcing a lot of paging.

Also, garbage collections are invoked depending on the frequency of
consing.  If modifying your database requires a lot of consing this can be
a problem.  Since Emacs doesn't do any compaction when it garbage collects
(except for strings), over time things become fragmentedm, virtual memory
behavior becomes horrible, and thus garbage collection takes longer and
longer and longer ...  (Of course, you could avoid this by using the
(often considered dangerous) setcar and setcdr functions.)

   I ask this because I can imagine GNU EMACS written such that the
   regular heap (for CONS cells and strings) is memory-resident but the
   buffers have a scheme of storing in temporary files when not active.

GNU Emacs relies entirely upon the operating system's support of virtual
memory to keep only active things in memory.  (That is, No, Emacs doesn't
do that.)

Separate, unrelated piece of advice:

Emacs provides hash tables, except that they're called "obarray"s.  You
can use the "intern" function to hash a string key into a Lisp symbol.
You can supply your own symbol table so that this new Lisp symbol isn't
visible to the rest of Emacs Lisp.  Simply store your database entry on
the symbol.

-- 
Joe Wells <jbw@bu.edu>