Path: utzoo!utgpu!jarvis.csri.toronto.edu!rutgers!apple!sun-barr!cs.utexas.edu!uunet!mcvax!hp4nl!botter!star.cs.vu.nl!biep
From: biep@cs.vu.nl (J A Biep Durieux)
Newsgroups: comp.lang.lisp
Subject: (Long) Code as data (Lisp's syntax - Answer to Stephen Knight).
Message-ID: <2727@ski.cs.vu.nl>
Date: 12 Jun 89 12:16:37 GMT
References: <1057@syma.sussex.ac.uk> <1350020@otter.hpl.hp.com>
Reply-To: biep@cs.vu.nl (J A Biep Durieux)
Organization: VU Informatica, Amsterdam
Lines: 239

In article <1350020@otter.hpl.hp.com>,
	sfk@otter.hpl.hp.com (Stephen Knight) writes:
>The discussion on the merits and demerits of Lisp syntax touched a couple
>of interesting points that I'd like to comment on.

>Macros in languages such as C or Pop11 are quite different beasts to that
>supported in Lisp(s).  Lisp does not make any distinction between parse and
>lexis so that macro processing in Lisp is parse tree procesing.  In other
>languages, macro processing is typically performed at the level of a flat,
>unstructured input stream.

Lisp works in three steps: read, eval and print.  The read step does the
lexical parsing (from string to parse tree), the eval step interprets the
resultung parse tree to an answer (in tree form), and the print step
translates the answer into string form again.

Connected with these three steps, Lisp has read macros, eval macros and
print macros, to steer these processes.  Besides that, one can replace
any of the procedures read, eval and print completely, if one wants to.
Many Lisps support several syntactic styles (parenthesised prefix notation,
mathematical notation, Pascal-like notation, etc.), several interpretation
styles (applicative, imperative-iterative, etc.), and several output styles
(normally matching the syntactic input styles).

So "macro processing" is really three different things in Lisp.

>As far as I am concerned, any language which makes me write
>    (+ x (* y z))	has made a rather dubious compromise.

Lisp doesn't "make you write" that. It just happens to be the default, just
like infix operators don't happen to be the default in Prolog.  One has to
declare them, just like one has to describe ones preferred syntax in Lisp.
Normally that boils down to just declaring that one wants infix notation,
if that is what one wants.
Any problems with e.g. [x+y*z], where the brackets are used to indicate the
switch in syntactic style (that is, if one writes in infix Pascal style
only, no brackets are needed)?

>Another example would be the overloading of #.  Why have #'(lambda ...) at 
>all?  Surely having (lambda ...) denote a function is a perfectly simple
>concept that fits well with the idea of a parse tree, and so on.

This is rather widely agreed on, and e.g. Scheme does things that way.

>Why denote true and false by #T and #F?  Why bother to denote them at all?
>One could simply use variables "true", "false", and "nil".  Why clutter up
>the syntax so pointlessly?  

Because one wants to be able to denote other data types, not just to be able
to write expressions that evaluate to them.  Another thing, of course, is
whether one should have many data types in the first place, but once they
are there they should be denotable.

>Why denote vectors with #( ... )?  Why not use different brackets?  How is
>#( ... ) any more tightly bound to the parse tree than, say, { ... } (as in
>Pop11).

Just to leave room for other syntactic conventions to work together with
the default.

>Indeed, why write lists as (list E1 .. En) rather than, say,
>[E1 ... En]?  After all, if one can have #( ... ) for vectors, why shouldn't
>lists have equal priviledge?

Again you are confusing denotation and evaluation.
(a b c) *denotes* a list with the elements a, b and c.
#(a b c) *denotes* a vector with the elements a, b, and c.

[outside evaluation context. Otherwise the above have to be quoted.]

(list 'a 'b 'c) *evaluates* to a fresh (not used before) list with the
	elements a, b, and c.

>Why are atoms quoted with a single quote at the start?  Why not have a quote
>at the end as well?  Most people are used to quotes matching.  Brackets match,
>after all.  Why not quotes?  String quotes match (of course). 

These are totally different entities.  Brackets and string quotes are lexical
annotations that help in parsing the input.  The quote is a read macro that
stands for a surrounding list with a first element "quote", that tells the
*evaluator* that the following S-expression (be it an atom, a list or a string)
should be taken as data.  A quote at the end would only introduce a possibility
for errors to occur: it might be put at the wrong place.

>The example of '#( ... )' was intended to provoke the observation that
>the choice of '#(' is completely arbitrary and therefore Lisp has a concrete
>syntax, at least in this instance.

But of course it has.  A default syntax, at least: "(" and ")" quite
arbitrarily denote list delimiters, "." denotes a pair half separator,
whitespace separates tokens, tokens consisting of digits only stand for
numbers, etc.

>Lisp has slowly but surely over the years acquired a concrete syntax whose
>relationship with the underlying parser tree is less and less direct.

Well, roughly a syntactic construct per data type. What is "less direct"
about that?

>It should be clear that there is an implicit closer for an open quote --
>namely the token boundary.  If the atom being quoted contains a token
>boundary in its text it needs quoting in a more conventional manner with
>string-like quotes.  As this is only an occasional requirement, Lisp has
>a second quoting convention rather than only one.  (It has at least
>three, of course.)

Quoting occurs at the parse-tree level (to be more exact: at the compilation
or evaluation instance), and the concept of "token" doesn't exist there.
I get the feeling you are mixing levels: at the same time you seem to be
talking about token delimiting (as with vertical bars) and evaluation
prevention (with the quote construction). Quote is like lambda or defun.

Look at these examples:

(define (a\ function x) (car x))	-->	|a function|

(|a function| (quote (a b c d e)))	-->	a

(\a\ \f\u\n\c\t\i\o\n ''x)		-->	quote

(1) |a function| is an atom, and its token representation is unimportant at
    evaluation time.
(2) Quoting may be done by writing "quote" explicitly.
(3) If the quote-readmacro is used, in the parse tree the word "quote"
    still appears.
(4) Token delimiting doesn't automatically quote the token

>The absurdity is that a quote is chosen for the implicit close convention and
>a vertical bar for the open-close convention.  The point here is that, for a
>beginner, Lisp is dominated by bracket-pairs which must be matched exactly.
>But almost immediately, the beginner is confronted with a quote (a close
>quote, in fact) that doesn't need opening.  The impression of arbitrary rules
>and poor choices is established almost immediately.

I can only interpret this if I assume that you indeed were confused about
lexical delimiting and evaluation prevention.
By the way, when you learned about function differentiation, did you feel
the opening quote was missing from f'(x) ?

>All and *only* literals would be quoted.  Thus where one would have written
>    (f 'a 'b 7)	one now has	(APPLY f (QUOTE a) (QUOTE b) (QUOTE 7))
>[So far, this should be familiar to anyone who has bumped into Lispkit Lisp.]  

All literals *may* be quoted already, and *only* literals can be quoted.
Self-evaluating expressions are an optional addition for ease of writing.

>The 'visionary' stage (which I freely admit is no more than a fantasy) is
>then to restructure Lisp standards to accomodate the idea of multiple
>syntaxes.

Have you had expreience with a good Lisp environment (e.g. Interlisp)?
They all have at least two syntaxes.

>Needless to say, the first and original syntax will be there, in all its
>glorious technicolor.  So this should satisfy Lispers who want to stick with
>the current concrete syntax.

Except the standardisation part, this is the current situation.  And see what
happens: almost everybody sticks with the old syntax.

>* If the language is intended for functional programming then I prefer
>  function application to be denoted by expression juxtaposition.  This 
>  doesn't work so well in Lisp because functions can take multiple
>  arguments.  So for this purpose, I think that the standard prefix form
>  that is familiar from mathematics and languages such as FORTRAN and Pascal
>  is appropriate.

Hey, going back to the olden days of M-expressions? What about EVALQUOTE?
No, seriously, I think there is a reason why S-expressions won from
M-expressions once they were introduced.

>* I think it is a point of good syntax design for the formal parameter list
>  to mirror the actual parameter list.  Thus I prefer DEFINE to DEFUN.

Agreed.  In fact, I think (define (f x y z) ...) will win over
(defun f (x y z) ...).  Lispers normally recognise something good when
they see it.  (Indeed, I think it's something good..)

>* The list syntax of Prolog is admirable because it elegantly integrates
>  bracketing with list appending without the need for quoting (as in Lisp
>  and Pop11) and *also* works in a functional context, unlike other Prologian
>  ideas.

But what would a literal list look like, then?  Remember that

(define (foo) '(a b c))
(eq (foo) (foo))		--> #t,

but
(define (bar) (list 'a 'b 'c))
(eq (bar) (bar))		--> #f.

How would you write these two in your notation?

>Examples:

>define append( x, y ) as
>    if x = [] then
>        y
>    else
>        [ x.head | append( x.tail, y ) ]
>    endif
>enddefine

This looks a lot like Interlisp's Clisp syntax.

>The next example is defining 'reverse'.  I'd do it like this.
>
>define fold( op, seed, list ) as
>    if list = [] then
>        seed
>    else
>        fold( op, op( list.head, seed ), list.tail )
>    endif
>enddefine
>
>define reverse( list ) as
>    fold( cons, [], list )
>enddefine

Why not define "fold" locally within "reverse"? You want the user to get
hold of (and possibly redefine) "fold" for some reason?

>And if I suspected the compiler was too stupid to optimise 'fold' into 
>an iterative format, I'd write the following, where -> is assignment from
>left to right.
>
>define fold( op, seed, list ) -> seed as
>    for i in list do
>        op( i, seed ) -> seed
>    endfor
>enddefine

Apart from the left-to-right assignment, I would surely take a look at Clisp.
-- 
						Biep.  (biep@cs.vu.nl via mcvax)
	Who am I to doubt the existence of God?   I am
	  only a simple man,  I already have trouble
	enough doubting the existence of my neighbour!