Path: utzoo!censor!geac!torsqnt!news-server.csri.toronto.edu!cs.utexas.edu!uunet!ogicse!borasky From: borasky@ogicse.ogi.edu (M. Edward Borasky) Newsgroups: comp.benchmarks Subject: Re: SPEC vs. Dhrystone Message-ID: <15577@ogicse.ogi.edu> Date: 3 Jan 91 17:58:23 GMT References: <44342@mips.mips.COM> <15379@ogicse.ogi.edu> <44353@mips.mips.COM> <1685@marlin.NOSC.MIL> <15546@ogicse.ogi.edu> <44465@mips.mips.COM> <44466@mips.mips.COM> Distribution: comp.benchmarks Organization: Oregon Graduate Institute (formerly OGC), Beaverton, OR Lines: 24 In article <44466@mips.mips.COM> mash@mips.COM (John Mashey) writes: >To my knowledge, no one has so far found anything quite like these things >in the SPEC benchmarks. The closest case is matrix300, where most of the >time is spent inside a small loop. This benchmark was included, >rightly or wrongly, as it at least thrashed the data cache around Correct me if I'm wrong, but the matrix300 is Jack Dongarra's 300- equation LINPACK benchmark, isn't it? If so, the inner loop is matrix multiply, unrolled 1, 2, 4, 8 and 16 ways. At least one compiler (FPS Model 500) recognizes the 1-way unrolled case as a matrix-vector multiply and calls in a hand-optimized matrix multiply routine. In this case, cache is not relevant because the hand-optimized routine knows about the memory hierarchy. On the subject of LINPACK cheating, we really ought to get Jack Dongarra involved in the discussion. Unfortunately, he HATES to talk about his experiences with vendors in this area. Every vendor had somebody who was ever vigilant for cases of OTHER vendors cheating; I filled that function for FPS at one time. >In some cases, about the only hack you could do would be to >generate a program that emitted a copy of the required output, and did nothing >else. Do you believe that vendors are working on this? If so, it would be relatively easy to defeat using techniques of public-key cryprography (which by the way is an interesting application for BENCHMARKING!!!!)