Path: utzoo!utgpu!attcan!uunet!lll-winken!lll-tis!helios.ee.lbl.gov!pasteur!agate!eos!ames!mailrus!uflorida!novavax!proxftl!bill From: bill@proxftl.UUCP (T. William Wells) Newsgroups: comp.std.c Subject: Re: preprocessing: character- or token-based? Keywords: preprocessor tokens ambiguity Message-ID: <623@proxftl.UUCP> Date: 26 Aug 88 04:44:54 GMT References: <812@polari.UUCP> Reply-To: bill@proxftl.UUCP (T. William Wells) Organization: Proximity Technology, Ft. Lauderdale Lines: 31 Summary: Expires: Sender: Followup-To: Distribution: : Without quoting exact passages, the Rationale seems to imply that the : Standard is written in such a way as to allow the preprocessor to be : either character based or token based. It seems to me that either that : implication is false, or else the Standard leaves things like: : : #define aplus +a : +aplus; /* one operator, or two? */ : : ambiguous in meaning. Can anyone point me (quoting exact passages :-) : to a place in the Standard which precisely defines what happens in this : case? It seems to me that the problem is that the Standard nowhere defines : the meaning of the part of translation phase 7 called "Preprocessing : tokens are converted into tokens." Two operators. The sentence is unambiguous. Especially since the statement was preceded by "White-space characters separating tokens are no longer significant." This would imply that the tokens are concatenated *without* white space being available to separate tokens, a clearly ridiculous conclusion. We can add to this, from 3.8.3 in the Rationale: "Preprocessing is specified in such a way that it can be implemented as a separate (text-to-text) pre-pass or as a (token-oriented) portion of the compiler itself." The first part of this sentence is asserting that the preprocessor can be designed as a program that reads text and writes text, it is not implying anything about the internal operation of the preprocessor. The second part implies that the preprocessor operates on tokens. --- Bill novavax!proxftl!bill