Path: utzoo!utgpu!news-server.csri.toronto.edu!rpi!zaphod.mps.ohio-state.edu!sdd.hp.com!elroy.jpl.nasa.gov!decwrl!sgi!shinobu!odin!patton.wpd.sgi.com!jmb
From: jmb@patton.wpd.sgi.com (Jim Barton)
Newsgroups: comp.sys.sgi
Subject: Re: re-cursing lex
Message-ID: <1991May2.012007.9975@odin.corp.sgi.com>
Date: 2 May 91 01:20:07 GMT
References: <9104290656.AA16311@karron.med.nyu.edu>
Sender: news@odin.corp.sgi.com (Net News)
Organization: Silicon Graphics Inc.
Lines: 56

In article <9104290656.AA16311@karron.med.nyu.edu>, Dan Karron@UCBVAX.BERKELEY.EDU writes:
|> From: Dan Karron@UCBVAX.BERKELEY.EDU
|> Newsgroups: comp.sys.sgi
|> Subject: re-cursing lex
|> Message-ID: <9104290656.AA16311@karron.med.nyu.edu>
|> Date: 29 Apr 91 06:56:24 GMT
|> Organization: The Internet
|> 
|> How do I get lex to: 
|> 
|> 1) read from a user provided buffer instead of a file ?
|> 
|> 2) Get lex to throw away output it can't match (instead of sinking OUTPUT
|> to /dev/null ?) Better still would be to write a catchall pattern that
|> would make an error string for stuff it could not match otherwise.
|> 
|> 3) Get lex to recurse, i.e., call lex again with input from another buffer.
|> 
|> I don't want to get involved with yacc, as that is too complicated for 
|> my simple mind.
|> 
|> All that I want to do is identify strings like
|> 
|> VARIABLE="value"
|> 
|> from a file or the environment string and
|> 
|> -VARIABLE="value" 
|> 
|> from the command line argument vector.
|> 
|> Once recognized, then parse out the value, or if in error, say something
|> about it.
|> 
|> I would like to take a uniform approach to reading values from a file,
|> environment string, and command line argument.

You can manage 1 and 3 in lex by redefining the standard input/output functions.
I actually think it's documented in that old hoary stuff from AT&T. The
functions are actually macros: "input", "output" and "unput". #undef these
at the top of your *.l file, and provide real functions to do what you want.
For instance, I wrote a lex program that could deal with "include" files
(even nested includes).

Number 2 could be handled pretty easy by including a final rule:

	. { }

I would suggest you use 'flex' instead. Flex is 1000% faster than lex, builds
faster parsers, has more features, is easier to use, doesn't have the weird
bugs in lex, and is free. The way you redefine input/output is much easier.
You can pick it up from any of the public domain archives.

-- Jim Barton
   Silicon Graphics Computer Systems
   jmb@sgi.com