Path: utzoo!utgpu!jarvis.csri.toronto.edu!mailrus!tut.cis.ohio-state.edu!cs.utexas.edu!uunet!mcvax!ukc!icdoc!syma!aarons
From: aarons@syma.sussex.ac.uk (Aaron Sloman)
Newsgroups: comp.lang.lisp
Subject: Code as data (Syntax, macros, run-time compilation)
Keywords: resistance to lisp
Message-ID: <1044@syma.sussex.ac.uk>
Date: 3 Jun 89 14:09:54 GMT
References: <31670@sri-unix.SRI.COM> <469@skye.ed.ac.uk> <1028@syma.sussex.ac.uk> <487@skye.ed.ac.uk> <11917@well.UUCP> <9636@polya.Stanford.EDU>
Organization: School of Cognitive & Computing Sciences, Sussex Univ. UK
Lines: 171

My original posting on the unfortunate effects of Lisp syntax produced a
number of comments and stirred up a discussion of code as data. This
message attempts to separate what is important about this in lisp from
what is superficial. In particular I try to show that the syntax of
lisp has nothing to do with the possibility of having macros and that
not all cases where code has to be treated as data require the original
source code to be manipulated.


nagle@well.UUCP (John Nagle) writes:

>      Syntax is an big issue for beginning programmers, but assumes much
> less importance once you know a few different languages.
My main point in raising this issue was that the syntax of Lisp has had
the effect of turning off a number of experienced programmers who might
otherwise have been tempted to get into AI tools and techniques.
This is consistent with the view that once you have got into Lisp
it is a fine language.

> ... Semantics and
> paradigm are much bigger issues when writing sizable programs.  As
> programs become larger, syntactical issues retreat in importance and
> issues such as namespace control and other modularity issues dominate.
Agreed - but syntax and in particular readability remain relevant when
someone has to maintain code written by others.

>      Most of the attempts to make LISP look like Pascal or one of its
> descendants result in a syntax that is more, rather than less, painful.

Painful for whom? Has this been investigated systematically? I can
well imagine that someone brought up on and/or very used to Lisp will
find the extra syntactic clutter objectionable whereas for others it
might make the language far more approachable (as Pop-11 has proved
to be for many experienced programmers.)

Have any tests been done on programmer productivity, reliability,
ease of maintenance over a longish period, using standard Lisp and
Lisp with a Pascal-like syntax? (I fear such tests would be very
expensive if done with sufficient thoroughness to give conclusive
evidence.)

shaff@Sesame.Stanford.EDU (Mike Shaff) writes:

> For people who need to write code that performs analysis of one
> form or another, simulates, etc a program the "code as data"
> quality of Lisp is critical to their effort.

I suspect that what is important is that there is a well defined
parse-tree or similar structure that can be derived from the source
code and manipulated by program. Whether the external form makes the
parsing totally trivial (because it is all based on parentheses) or
not is probably far less important. (But see comments below on
manipulating source code vs manipulating structured procedure
objects).

mthome@bbn.com (Mike Thome) writes:

> In article <11917@well.UUCP> nagle@well.UUCP (John Nagle) writes:
> >     Most of the attempts to make LISP look like Pascal or one of its
> >descendants result in a syntax that is more, rather than less, painful.
> For example, take a look at Logo...

I am amazed that anyone regards Logo as an attempt to make Lisp look
like Pascal! One of the flaws in the design of at least early
versions of Logo (I've not looked at current versions) appears to
have been the assumption that giving people fewer syntax forms to
learn and fewer punctuation symbols (no parentheses, commas,
semi-colons etc) makes the language easier to learn. As if the
problems of a learner have to do primarily with how much typing your
fingers have to do, as opposed to how much parsing your brain has to
do. We seriously considered Logo as a teaching language for AI in the
mid 70s and rejected it in favour of Pop. Had Pop not been available
we'd have preferred Lisp to Logo.

> .... When I was learning lisp, at first
> it was its syntactic consistancy which was probably the most attractive
> attribute.
Yes - there are many people who react like that, especially people
with mathematical aptitude and training. However, there are also
many (including some experienced programmers) who are put off by the
cognitive effort that is required because of the syntactic economy.

>
> >On the other hand, the fact that data and programs have the same
> >representation in LISP really doesn't seem to be used all that much
> >any more.  It was felt to be terribly important at one time, but today,
> >it just doesn't seem to be a big issue.
> >					John Nagle
> I must disagree here - any macro (including most setf expanders) will
> depend on the similarity between data and program......

However, this does not require something like Lisp syntax. E.g.
Pop-11 (following Pop-2) allows macros. The input stream is a
dynamic list of text items (words, strings, numbers etc) and a macro
is just the name of a procedure that reads in a portion of that stream
changes it and then puts back the result onto the front of the list.
If it contains no macros that is what will be compiled in place
of the origina,. Lisp syntax makes the reading and rearranging easier.
Pop syntax gives more freedom for syntactic extensions - e.g.
expressions and statements don't have to begin and end with parentheses.

So "similarity between data and program" as in Lisp is not necessary
for macros.

>.......Personally, I've been
> working on a number of different projects which build (sometimes compile)
> routines at runtime.  While this might even be unusual, lisp is, after
> all, one of the very few (only mainstream?) language which can do this -
> I have a hard time imagining what it would be like to try to implement
> one of these systems in a "conventional" language.

This is a standard facility in Pop-11. In fact, Pop-11 procedures
are provided for planting instructions to the Poplog virtual
machine, which can then be compiled. This enables Poplog Pop-11
to be extended with languages like Common Lisp, Prolog, Scheme and ML
without requiring them to have anything remotely like the syntax of
Pop-11, and without requiring them to be interpreted: they all compile
to machine code.

rosenstn@hi3.ACA.MCC.COM (Mark Rosenstein) writes (in response
to John Nagle):

> Ummmm. The only datapoint I have is my own work, and work in the 5
> or 6 projects here at work that I know about. I would say all of them
> use the fact that data and program have the same representation. I
> would be so bold as to exagerate that the only interesting lisp programs
> are ones that generate and manipulate structure as programs. [Then compile
> 'em and use 'em, of course]. Of course elsewhere it may be different.
> I find this especially true in object oriented systems like CLOS and
> flavors, where a lot of what happens is that the program is happily
> creating new classes and methods for those classes and then using
> them.

In Pop-11 a lot of program manipulation makes use of the fact that
you can "partially apply" a procedure to some data (including other
procedures) to create a new procedure. Partial application creates
structures that can either be RUN as procedures (possibly requiring
extra arguments) or analysed as data (you can examine or alter the
procedure part, or the data-part). These entities tend to run much
faster than interpreted code, but to have essentially the structure you
require for analysing a procedure at a level of abstraction that ignores
the fact that you have built the procedure out of a particular piece
of code.

E.g. -applist- is a procedure that takes a list L and a procedure
P and applies P to every element of L. If -proc- is a procedure, then
applist(%proc%) is a new procedure formed by partially applying
-applist- to -proc-, which can be applied to a list, in which case it
will apply -proc- to every element. You can either RUN this new procedure
or treat it as structured data and access the components (applist and
proc).

The Lisp way of doing things makes you think in terms of the actual code
rather than in terms of the composition of procedures and data, which, I
suspect, is what is mostly required. However, if you really must work at
the level of code, Pop-11 allows programs to build lists of code, then
compile and run them. So you get both options, though manipulating Pop
code by program is harder than manipulating Lisp code.

There are some examples illustrating all these points in our book
R. Barrett, A. Ramsay and A. Sloman
    POP-11: A Practical Language for AI,
    Ellis Horwood and John Wiley, 1985, reprinted 1986.

although Pop-11 has evolved since this was written. It now includes
full lexical scoping and other extensions.

Aaron Sloman,
School of Cognitive and Computing Sciences,
Univ of Sussex, Brighton, BN1 9QN, England
    INTERNET: aarons%uk.ac.sussex.cogs@nsfnet-relay.ac.uk