Path: utzoo!utgpu!jarvis.csri.toronto.edu!rutgers!cs.utexas.edu!uunet!odi!dlw
From: dlw@odi.com (Dan Weinreb)
Newsgroups: comp.lang.lisp
Subject: Re: Code as data (Syntax, macros, run-time compilation)
Message-ID: <378@odi.ODI.COM>
Date: 5 Jun 89 04:26:37 GMT
References: <31670@sri-unix.SRI.COM> <469@skye.ed.ac.uk> <1028@syma.sussex.ac.uk> <487@skye.ed.ac.uk> <11917@well.UUCP> <9636@polya.Stanford.EDU> <1044@syma.sussex.ac.uk>
Reply-To: dlw@odi.com
Organization: Object Design Inc., Burlington, MA
Lines: 64
In-reply-to: aarons@syma.sussex.ac.uk's message of 3 Jun 89 14:09:54 GMT

In article <1044@syma.sussex.ac.uk> aarons@syma.sussex.ac.uk (Aaron Sloman) writes:

   However, this does not require something like Lisp syntax. E.g.
   Pop-11 (following Pop-2) allows macros. The input stream is a
   dynamic list of text items (words, strings, numbers etc) and a macro
   is just the name of a procedure that reads in a portion of that stream
   changes it and then puts back the result onto the front of the list.
   If it contains no macros that is what will be compiled in place
   of the origina,. Lisp syntax makes the reading and rearranging easier.
   Pop syntax gives more freedom for syntactic extensions - e.g.
   expressions and statements don't have to begin and end with parentheses.

What Pop-11 macros are given is like the output of the "lexer" part of
a compiler, i.e. the input to the "parser" part, whereas what Lisp
macros are given is like the output of the "parser" part.  Lisp macros
deal in the equivalent of "parse trees".  If you want to write a macro
that extends the language in an interesting way, such as adding a new
iteration construct, adding a special form to define "rules" or
something, etc., it is far easier if you are given the parser's
output.  To do the equivalent in Pop-11, you'd have to essentially
write your own parser (possibly simpler than that of a full compiler,
but it would have to know a lot about the language).

Lisp macros derive their tremendous power and utility from the way
they transform one "parse tree" into another.  To be able to write a
Lisp macro, the programmer must be familiar with the format of the
parse tree.  Lisp's syntax makes this very easy, because the
correspondance between what you see on the page and how the parse tree
is organized is very simple.  Yes, you could still have Lisp macros if
a Pascal-like syntax were used, but it would make macro-writing more
difficult.  This is a tradeoff that should be considered carefully
by people who are consider alternative Lisp syntaxes.

You mentioned Pop-11's ability to let you use some character other
than parentheses for lists, and things like that.  Lisp has this, too.
It's a distinct facility, known as "reader macros", that let you do
transformations at the lexical level rather than the parse-tree level.
Doing this kind of thing has its drawbacks.  One is that the text
editors that currently exist need to be taught about the reader
macros, so that their language-understanding command will work.
Another drawback is that reader macros obscure the correspondance
between what you see on the page and how the parse tree is organized,
so macro-writers need to be keep in mind what the reader macros are
doing; the more you have to keep in your mind, the harder it is to
program.  For these and various other reasons, good Lisp programmers
only use reader macros sparingly, when there's a lot of benefit to
be had.  But they're there if you need them, and you can use them
as much as you want.  (The Lisp single-quote is typically implemented
as a very simple reader macro.)

By the way, the idea of alternative Lisp syntaxes is not new.  There
was once an attempt to define such a thing, at MIT, way before my
time, called "Lisp 2".  It was so long ago that I have never been able
to find any real documentation on it.  I believe the effort to define
it was abandoned for some reason.  One of the subsequent attempts at
MIT was defined and implemented successfully.  It was called CGOL,
written by Vaughn Pratt, I think sometime around 1976.  There is an
MIT AI Lab memo that describes it, and the source code might very well
be available from the AI Lab, for all I know.  CGOL apparently never
caught on with any user community.  Obviously, that fact by itself
proves nothing about its worthiness, value, etc.  I just wanted to
make the point that the idea has been explored in the past, at least
two times, and therefore probably more times than that.  Just thought
you might be interested.