Path: utzoo!attcan!uunet!know!sdd.hp.com!zaphod.mps.ohio-state.edu!usc!apple!rutgers!psuvax1!husc6!spdcc!ima!esegue!compilers-sender
From: djones@megatest.uucp (Dave Jones)
Newsgroups: comp.compilers
Subject: Re: Question About YACC's Memory Management
Keywords: yacc
Message-ID: <14252@goofy.megatest.UUCP>
Date: 21 Oct 90 00:19:32 GMT
References: <1990Oct18.213224.21195@csrd.uiuc.edu>
Sender: compilers-sender@esegue.segue.boston.ma.us
Reply-To: djones@megatest.uucp (Dave Jones)
Organization: Megatest Corporation, San Jose, Ca
Lines: 45
Approved: compilers@esegue.segue.boston.ma.us

In article <2371@kiwi.mpr.ca>, andrea@mprgate.mpr.ca (Jennitta Andrea) writes:
> 
>   I have a parser which is intended to be called repeatedly from a driver
> program.  ...
> 
>   I have typed the value stack to contain pointers to characters.  I
> malloc the memory required for each token in lex (assigning yylval to
> point to that block of memory) before returning the token to yacc.  It
> appears that this memory is not cleaned up when yacc frees the value stack.
> 
>   Am I required to explicitly free each token once the parser has reduced
> a rule?  ...
> 

Yes.

But you probably don't really want to use malloc directly for this purpose,
anyway.

One way is to write or borrow an expandable string-table package.
When you are through with the strings, free them all at one go.

But why copy the tokens at all? A technique I have used when extreme
speed was wanted is to read the entire input file into a buffer at the
start. Then a token is coded for with a structure containing a pointer
into the file-buffer, and a character count. The structures, being all
the same size, can be allocated in blocks of a thousand or so, and kept
in free-lists for re-use rather than returned with free().

Actual example follows. The pointer into the buffer and the character
count are the second and third fields respectively.

typedef struct token
{ int    type;          /* as per gram.y and y.tab.h */
  char*  token_image;   /* pointer to token within line  */
  int    length;        /* length of token */
  char*  line_image;    /* \n terminated line containing token */
  int    line_num;      /* line number within source file */
  struct src *src_file; /* file containing the token */
  int    hval;          /* hash-value for identifiers... */
}
Token;
-- 
Send compilers articles to compilers@esegue.segue.boston.ma.us
{ima | spdcc | world}!esegue.  Meta-mail to compilers-request@esegue.