Path: utzoo!mnetor!uunet!husc6!necntc!ima!johnl From: lbl-pistachio!vern@lbl-rtsg.arpa (Vern Paxson) Newsgroups: comp.compilers Subject: Re: replacements for LEX: flex, ... , Rex Message-ID: <938@ima.ISC.COM> Date: 2 Apr 88 03:53:59 GMT Sender: johnl@ima.ISC.COM Reply-To: Vern Paxson Lines: 60 Approved: compilers@ima.UUCP Josef Grosch describes Rex, a scanner generator similar to lex, and gives some timings comparing its perfomance with lex's: > Scanners generated by Rex are more than 4 times faster and up to 5 > times smaller than LEX-generated ones. It is possible to reach a speed > of 180,000 to 195,000 lines per minute on a MC 68020 processor (includ- > ing input from file).... > .... I would like to hear some numbers from "flex". Here they are for a C tokenizer which includes keywords. Generation times: lex 83.0 secs flex 3.9 flex -cfe 7.1 # uncompressed table, equivalence classes flex -cf 15.0 # uncompressed table, no equivalence classes Scanner object file sizes: lex 41.0K bytes flex 9.4K flex -cfe 49.6K flex -cf 126.5K Running times on a 28,088 line input (685K characters): lex 29.8 secs flex 19.3 flex -cfe 9.0 flex -cf 7.8 The timings were made on a Sun 3/60. All times are user + system CPU time, and don't include hashing of identifiers. Summary: For about the same sized scanner, you get a factor of 3 in performance. For a 30% faster scanner, you get a scanner 1/4th the size, and it's generated in 1/20th the time. For a scanner that's 3 times larger, you get a factor of 3.8 in performance. Rex sounds very good - taken separately, various options of flex perform in about its league, but flex can't match its combined performance. This is due in part to flex's compressed scanners not having been optimized for speed nearly as much as the full table scanners. What is Rex's availability? Vern Vern Paxson vern@lbl-csam.arpa Real Time Systems ucbvax!lbl-csam.arpa!vern Lawrence Berkeley Laboratory (415) 486-6411 -- Send compilers articles to ima!compilers or, in a pinch, to Levine@YALE.EDU Plausible paths are { ihnp4 | decvax | cbosgd | harvard | yale | bbn}!ima Please send responses to the originator of the message -- I cannot forward mail accidentally sent back to compilers. Meta-mail to ima!compilers-request