Path: utzoo!utgpu!jarvis.csri.toronto.edu!mailrus!cs.utexas.edu!wuarchive!mit-eddie!mit-amt!adamk
From: adamk@mit-amt.MEDIA.MIT.EDU (Adam Kao)
Newsgroups: comp.lang.c
Subject: What does "a---b" mean?
Message-ID: <1303@mit-amt.MEDIA.MIT.EDU>
Date: 3 Jan 90 07:01:00 GMT
Reply-To: adamk@media-lab.media.mit.edu.UUCP (Adam Kao)
Organization: MIT Media Lab, Cambridge MA
Lines: 41

I've been thinking about a C interpreter and ran into this problem.
Harbison & Steele says the lexical scanner always makes the largest
token, so I assume the example from the subject line would be tokenized
as "(a--) - b", and in general "a---...---b" would come out as
"(((a--)--)...--)-b" or something (ignoring what that might mean).

But it's not clear to me that this is consistent with the precedence
and associativity rules (mainly because I have trouble understanding
them!).  Is it actually true that the lexical scanner is defined to be
"dumb" and MUST ignore syntax while tokenizing?  Then "a--b" would
always generate a compile error, right?  (Waitaminute!  This explains
that problem my friend was having . . .)

This is further complicated by the requirement that compound
assignment operators actually be considered as two separate tokens
during the scanning phase ("a - = b" is "a -= b" . . .) but the tokens
are combined into one token before being sent off to the syntax
analyzer.

More generally I'm not sure I understand how C handles ambiguity with
multiple-character tokens.  To summarize, my understanding is:

1. The lexical analyzer scans left to right making the biggest token possible.
2. Compound assignment tokens (ONLY) can be read as two consecutive
   tokens (ie separated by whitespace ONLY) that are IMMEDIATELY
   joined into one token.
3. Remaining ambiguity is resolved by simple (hah!) precedence and
   associativity rules.  All remaining ambiguities can be resolved by
   these rules, which essentially answer the questions "When do
   parentheses enclose this operator?" (precedence) and "Which
   operator does this operand go with?" (associativity).  In
   particular, the questions "Which operand does this operator go
   with?" and "What operator(s) is this?" never arise or are irrelevant.

Is this correct?  Is it complete?

Any other suggestions or "gotcha's" about C interpreters are welcome.

Thank you for your time.

Adam