Path: utzoo!censor!geac!torsqnt!news-server.csri.toronto.edu!cs.utexas.edu!swrinde!zaphod.mps.ohio-state.edu!mips!winchester!mash From: mash@mips.COM (John Mashey) Newsgroups: comp.benchmarks Subject: Re: SPEC vs. Dhrystone Message-ID: <44466@mips.mips.COM> Date: 3 Jan 91 17:17:54 GMT References: <44342@mips.mips.COM> <15379@ogicse.ogi.edu> <44353@mips.mips.COM> <1685@marlin.NOSC.MIL> <15546@ogicse.ogi.edu> <44465@mips.mips.COM> Sender: news@mips.COM Reply-To: mash@mips.COM (John Mashey) Distribution: comp.benchmarks Organization: MIPS Computer Systems, Inc. Lines: 49 In article mccalpin@perelandra.cms.udel.edu (John D. McCalpin) writes: >>>>>> On 3 Jan 91 06:18:59 GMT, mash@mips.COM (John Mashey) said: >mash> at least with Whetstone, Dhrystone, and LINPACK. > >So what optimizations have been performed on the LINPACK 100x100 code >that are "absolutely useless" in real life? As reported in Patterson&Hennessy, page 47, "An extreme example of such targeted engineering employed compiler optimizations that were benchmark sensitive. ...a person at one company used a preprocessor that scanned the text for keywords to try to identify benchmarks by looking at the name of the author and the name of a key subroutine." Although it doesn't say so in P&H, this was a search for "DONGARRA". to be fair, one might call this outright cheating rather than a real optimization. Certainly, it is at the extreme useless end of the scale of: a) Hunt for Dongarra b) Hunt for the inner loop, and character-match, and if so, generate optimal code. c) Do all of the analysis like you should, and generate optimal code. For Whetstone, the classic one is recognizing a trigonometric identity, and thus avoiding a fairly expensive computation; for Dhrystone, it is the strcpy stuff, or earlier, removal of dead code. ------- Reinhold Weicker has published some good analyses of common benchmarks; there was a talk for Slater's Microprocessor Forum last Spring, and one for RISC'90 in Karlsruhe in November. I think there's one in the current (or immediate past) issue of COMPUTER: maybe somebody can cite the exact reference, as I don't have it handy. ------- To my knowledge, no one has so far found anything quite like these things in the SPEC benchmarks. The closest case is matrix300, where most of the time is spent inside a small loop. This benchmark was included, rightly or wrongly, as it at least thrashed the data cache around, and the inner loop is representative of some real calculations; certainly it was argued about a lot... For most of the SPEC benchmarks, even if you knew that you were looking at that benchmark, it wouldn't do you much good. I.e., exactly what good does it do to know you're compiling gcc? In some cases, about the only hack you could do would be to generate a program that emitted a copy of the required output, and did nothing else. I believe this would be fairly easy to discover :-) -- -john mashey DISCLAIMER: UUCP: mash@mips.com OR {ames,decwrl,prls,pyramid}!mips!mash DDD: 408-524-7015, 524-8253 or (main number) 408-720-1700 USPS: MIPS Computer Systems, 930 E. Arques, Sunnyvale, CA 94086