Xref: utzoo comp.sources.d:3996 comp.os.vms:17215
Path: utzoo!utgpu!jarvis.csri.toronto.edu!rutgers!bellcore!att!occrsh!jct!
From: jct@jct.UUCP (jct)
Newsgroups: comp.sources.d,comp.os.vms
Subject: Re: Flex (latest 2.1 beta) on VMS
Summary: FLEX bug on VMS (and anything else)
Message-ID: <291@jct.UUCP>
Date: 29 Aug 89 22:21:30 GMT
References: <13603@bloom-beacon.MIT.EDU> <1664@naucse.UUCP>
Organization: Kerr-McGee, Okla. City
Lines: 61

In article <1664@naucse.UUCP>, jdc@naucse.UUCP (John Campbell) writes:
> From article <13603@bloom-beacon.MIT.EDU>, by scs@adam.pika.mit.edu (Steve Summit):
> : In article <1629@naucse.UUCP> jdc@naucse.UUCP (John Campbell) writes:
> :>The new flex reads a large chunck at a
> :>time.  With VMS STREAM-LF files this works just fine--but with "normal"
>
> Stuff deleted
>
> : This sounds like a bug in flex.  If I understand the complaint
> 
> I'm the original poster of the first article.  To date I have not proven
> that there is a bug in flex.  I still believe this is true, but I haven't
> had the time to make the obvious tests in other environments.  I still
> believe that yyunput() is in error in some way, but I would like to have
> some follow up information before I bug Vern Paxson with my worries.
> 
> Stuff deleted
>

I noticed the same problems. I traced it to a macro substitution occuring
at the beginning of a read() buffer. The substitustion uses yyunput() to do
the actual change. Since on VMS this is the beginning of a line it shows up 
more easily than on UNIX. I say its a flat out bug in yyunput(), I changed 
to the following (which is less CPU efficient but works):

#ifdef __STDC__
void yyunput (int c, register char *yy_bp)
#else
void yyunput (c, yy_bp)
  int           c;
  register char *yy_bp;
#endif
{
  register char *yy_cp = yy_c_buf_p;

  *yy_cp = yy_hold_char;		/* undo effects of setting up yytext */
  if (yy_cp < yy_ch_buf + 2)
    { 					/* need to shift up to make room */
      register int number_to_move = yy_n_chars + 2;	/* +2 for EOB chars */
      register char *source = &yy_ch_buf [number_to_move];
      register char *dest = source + 1;

      while (source > yy_ch_buf)
	*--dest = *--source;
      yy_cp++;
      yy_bp++;
      yy_n_chars++;
      if (yy_cp < yy_ch_buf + 2)
	YY_FATAL_ERROR ("flex scanner push-back overflow");
    }
  if (yy_cp > yy_bp && yy_cp [-1] == '\n')
    yy_cp [-2] = '\n';
  *--yy_cp = c;
  YY_DO_BEFORE_ACTION;			/* set up yytext again */
}

That is instead of moving the whole buffer up to the end, just move up
to make room for 1 more character. I've had no problems since.

John Tompkins
occrsh!jct!jct