Path: utzoo!utgpu!watmath!clyde!ima!think!barmar
From: barmar@think.COM (Barry Margolin)
Newsgroups: comp.lang.lisp
Subject: Re: The best way to save data
Keywords: save, load, lisp, Database
Message-ID: <33585@think.UUCP>
Date: 16 Dec 88 08:10:19 GMT
References: <627@wucs1.wustl.edu>
Sender: news@think.UUCP
Reply-To: barmar@kulla.think.com.UUCP (Barry Margolin)
Organization: Thinking Machines Corporation, Cambridge MA, USA
Lines: 31

The easiest way to program it is by using WRITE and READ (not LOAD --
that expects the file to contain executable forms, not just random
data).  If portability is important, this is probably your best bet.
But the generality comes at a price: numeric data is not very compact,
and parsing is expensive.  Writing the data is usually pretty quick,
though (just make sure you specify :PRETTY NIL :CIRCLE NIL, or WRITE
will have to make multiple passes over the data).

If speed is more important than portability, though, you'd probably
want to write more specialized routines.  How easy this is will depend
on your data.  The expensive part of READ is that it must parse the
data in order to determine its type.  You can speed things up by
writing a single-byte type code followed by the data.  For
variable-length data you could include a length byte after the type
code.  You can then have type-specific input routines that read the
object as efficiently as you can devise.  Symbols could be written
just as their print names, without escape prefixes (since the length
code tells you exactly where the symbol name ends).  Small integers
could be encoded in a single byte, and longer integers could easily be
written with a length code followed by that many bytes of the actual
data, rather than encoding it in decimal as WRITE does.  You can even
handle data types that aren't ordinarily readable by READ.

This is quite a bit of work, but if you'll be writing and reading lots
of data it can be worthwhile.

Barry Margolin
Thinking Machines Corp.

barmar@think.com
{uunet,harvard}!think!barmar