Path: utzoo!censor!geac!torsqnt!news-server.csri.toronto.edu!clyde.concordia.ca!uunet!microsoft!t-rayc
From: t-rayc@microsoft.UUCP (Raymond CHEN)
Newsgroups: comp.text.tex
Subject: Re: Verbatim environment in macro argument
Message-ID: <55769@microsoft.UUCP>
Date: 11 Jul 90 16:33:34 GMT
References: <MERNST.90Jul9133647@emu.lcs.mit.edu> <MERNST.90Jul10132554@wren.lcs.mit.edu>
Reply-To: t-rayc@microsoft.UUCP (Raymond CHEN)
Distribution: comp
Organization: Microsoft Corp., Redmond WA
Lines: 46

In article <MERNST.90Jul10132554@wren.lcs.mit.edu> mernst@theory.lcs.mit.edu (Michael Ernst) writes:
>Can I somehow re-scan a piece of text that has already been
>tokenized to re-classify each of the tokens, perhaps giving them new
>catcodes?

Taking inspiration from the top of page 382, with the same caveats:

\newcount\oldescapechar
\newwrite\tokout
\newread\tokin

\def\retokenize#1{%
	\immediate\openout\tokout=toktmp
	{\escapechar=\oldescapechar
	\immediate\write\tokout{\expandafter\strip\meaning#1}}%
	\immediate\closeout\tokout
	\openin\tokin=toktmp
	\read\tokin to#1%
	\closein\tokin
}
\def\strip#1>{}

% ! will be the escape character, and () will be the grouping characters.

\def\mytok{!message(Now isn't that special.)}

% Okay, now we change catcodes like a madman.

\oldescapechar=\escapechar
\escapechar=`\!
\catcode`\!=0
!catcode`!\=12

!catcode`!(=1
!catcode`!)=2
!catcode`!{=12
!catcode`!}=12

!retokenize!mytok

% Now I execute the new stuff: 
!mytok
!end

% Don Hosek can probably find a less kludgy way of doing it.
% (Alternate reply path:  raymond@math.berkeley.edu)