Path: utzoo!attcan!uunet!cs.utexas.edu!hellgate.utah.edu!cdr.utah.edu!moore
From: moore%cdr.utah.edu@cs.utah.edu (Tim Moore)
Newsgroups: comp.lang.lisp
Subject: Re: Building a string, char-by-char -- more details (HELP!)
Message-ID: <1990Oct2.094857.8540@hellgate.utah.edu>
Date: 2 Oct 90 15:48:57 GMT
References: <51809@brunix.UUCP> <2990@idunno.Princeton.EDU>
Organization: University of Utah CS Dept
Lines: 69

In article <2990@idunno.Princeton.EDU> eliot@phoenix.Princeton.EDU (Eliot Handelman) writes:
>In article <51809@brunix.UUCP> dkb@cs.brown.edu (Dilip Barman) writes:
>> [How do I turn a string sentence into a list?]
>If READ-FROM-STRING acted reasonably, you could use it like this:
>
>(defun string->list (string)
>  (let ((words '()) (index 0))
>    (loop
>     (multiple-value-bind (word next-index)
>	 (read-from-string string nil nil :start index)
>       (setq index next-index)  
>       (if word
>	   (push word words)
>	   (return (nreverse words)))))))

>Unfortunately, READ-FROM-STRING throws an error if it sees read-macros,
>like commas. The solution is to write your own version, which reads
>characters from a string (using SCHAR), preprocesses special characters
>(like comma), detects the end of the word, then hands the string and
>indicies to SUBSEQ, which operates on the string. Intern the result,
>push on a list, NREVERSE when done and voila! 

Rather than rewrite the reader, the solution is to do some readtable
hacking. For example:

(defvar sentence-read-table (copy-readtable))
(set-macro-character #\. #'(lambda (stream char) '|.|)
		     nil sentence-read-table)
(set-macro-character #\, #'(lambda (stream char) '|,|)
		     nil sentence-read-table)
;;; ... and so on.

(defun string->list (string)
  (let ((words '())
	(index 0)
	(*readtable* sentence-read-table))
    (loop
     (multiple-value-bind (word next-index)
         (read-from-string string nil nil :start index)
       (setq index next-index)
       (if word
           (push word words)
           (return (nreverse words)))))))

(string->list "Alas poor Yorick, I knew him well Horatio.")
(ALAS POOR YORICK |,| I KNEW HIM WELL HORATIO |.|)

If you are willing to sacrifice a character, a 4 line hack (plus
readtable initialization) that does the same thing is:

(defun string->list2 (string)
  (let ((*readtable* sentence-read-table))
    (with-input-from-string (s (concatenate 'simple-string string "`"))
      (read-delimited-list #\` s))))

>It really is a pain to to this in CL. It was so much easier in the
>old Franz Lisp, because strings and atoms were identical. 
>
>--eliot

It's not that hard in Common Lisp. CL's extensive macro character
syntax can get in the way, but a one-time setup of a new read table
gets around this. In some sense strings and symbols are equivalent, as
many CL string functions will take a symbol argument and coerce it to
a string. 

Tim Moore                    moore@cs.utah.edu {bellcore,hplabs}!utah-cs!moore
"Ah, youth. Ah, statute of limitations."
		-John Waters