Path: utzoo!utgpu!news-server.csri.toronto.edu!cs.utexas.edu!uunet!munnari.oz.au!goanna!ok
From: ok@goanna.cs.rmit.oz.au (Richard A. O'Keefe)
Newsgroups: comp.lang.misc
Subject: Re: Re^8: Some things that pointer-less languages can't do efficiently
Message-ID: <4180@goanna.cs.rmit.oz.au>
Date: 1 Nov 90 09:04:36 GMT
References: <26739:Oct1023:44:2690@kramden.acf.nyu.edu> <65450@lanl.gov> <423@data.UUCP>
Organization: Comp Sci, RMIT, Melbourne, Australia
Lines: 25

In article <423@data.UUCP>, kend@data.UUCP (Ken Dickey) writes:
> peter>No, the term "token" has a meaning in the larger world of CS: a token is
> peter>an opaque object that refers to another object. In the context of compilers
> peter>it's a small easily-handled object that refers to a string of characters.

"peter" is right.

> I have looked through a number of standard CS texts and
> have been unable to find a definition of "token" other than an a
> meaningless (but unique) marker used in token-ring networks or as a
> syntactic unit in computer language analysis.  I ask again, what are
> the references which support your definition?

You could start by looking in almost any book on dataflow machines.
You could look at Allen, "Natural Language Understanding", where we find
"A new class of nodes [in a semantic network], called TOKENS, is
introduced to represent particular objects and particular actions".
This is almost exactly the sense in which "peter" used the word.
Then there are place/transition nets.

A few more textbooks may be in order.

-- 
The problem about real life is that moving one's knight to QB3
may always be replied to with a lob across the net.  --Alasdair Macintyre.