Path: utzoo!utgpu!news-server.csri.toronto.edu!cs.utexas.edu!swrinde!zaphod.mps.ohio-state.edu!usc!apple!agate!shelby!neon!lucid.com!eb From: eb@lucid.com (Eric Benson) Newsgroups: comp.lang.lisp Subject: Re: Compiling lexical closures in Common Lisp Summary: Incompatible closure representations Message-ID: <2241@heavens-gate.lucid.com> Date: 12 Jun 90 21:53:54 GMT References: Distribution: comp Organization: Lucid, Inc. Menlo Park, CA Lines: 80 In article , tim@cstr.ed.ac.uk (Tim Bradshaw) writes: > This is probably a naive question, so I apologise in advance... > > In Common Lisp, if you do the following, for instance: > > (setf (symbol-function 'foo) > (let ((foo 0)) #'(lambda () > (incf foo)))) > > then foo has a lexical closure as its symbol-function. This is all > well and good. But if I say "(compile 'foo)", in all the Common Lisps > I have tried, the compiler barfs. On the other hand, if I stick this > in a file and compile the file, then, again in all the lisps I have > tried, I end up with a compiled closure where, significantly, the code > of the function has been compiled. (The Common Lisps I have tried are > Franz Allegro on Sun4s, kcl on Sun4s and Xerox Medley.) > > Now I reckon that any symbol that has a legal function cell should be > compilable for the sake of uniformity if nothing else; and I further > reckon that I shouldn't need to write the thing out to a file to > compile it. But obviously this isn't the case, and it would be > interesting to know why! This isn't really naive, it's a subtle problem and one that *could* be fixed, but hasn't been because the level of effort far outweighs the benefits. The basic problem is that in most CL implementations compiled closures have an entirely different representation from interpreted closures. In this case you have created an interpreted closure with a reference to the variable FOO. This is represented using a pointer to a piece of the interpreter environment that contains the "value cell" for FOO as it was created when you first evaluated the LET binding. The interpreter knows how to find that value cell when it sees the reference to the variable. If the entire form is compiled, the compiler creates its own value cell for FOO (usually considerably more efficient than the interpreter's representation) and uses that for the reference inside the closure. Now, when you evaluate the form above and then try to compile *just the lambda expression*, you are asking the compiler to use the interpreter's value cell for FOO, as created by the LET binding in the first place. Naturally, this is possible (hey, it's software, anything is possible), but it would require a change at the very heart of the compiler and would introduce a dependency between the compiler and interpreter that currently doesn't exist. If someone tried to change the environment representation in the interpreter, the compiler would break! I actually wrote the code in Lucid Common Lisp to deal with situation. If you try to compile an interpreted closure with a non-empty environment, you get the following: > (setf (symbol-function 'foo) (let ((foo 0)) #'(lambda () (incf foo)))) # > (compile 'foo) FOO has a non-empty lexical environment. References to this environment will not be compiled correctly. Do you wish to try compiling FOO anyway? (Y or N): n NIL > This has not been a major issue for customers (at least that I know about) because there is always some other way to handle the problem, either by compiling a file or by wrapping a lambda around the entire expression, e.g. > (funcall (compile nil #'(lambda () (setf (symbol-function 'foo) (let ((foo 0)) #'(lambda () (incf foo))))))) [Now FOO is compiled.] eb@lucid.com Eric Benson 415/329-8400 x5523 Lucid, Inc. Telex 3791739 LUCID 707 Laurel Street Fax 415/329-8480 Menlo Park, CA 94025