Path: utzoo!utgpu!news-server.csri.toronto.edu!cs.utexas.edu!usc!julius.cs.uiuc.edu!apple!agate!shelby!eos!eugene
From: eugene@eos.arc.nasa.gov (Eugene Miya)
Newsgroups: comp.benchmarks
Subject: Re: PD suite of benchmarks
Message-ID: <7630@eos.arc.nasa.gov>
Date: 29 Nov 90 18:53:22 GMT
References: <1990Nov29.073135.21264@engin.umich.edu>
Reply-To: eugene@eos.UUCP (Eugene Miya)
Distribution: na
Organization: NASA Ames Research Center, Calif.
Lines: 44

In article <1990Nov29.073135.21264@engin.umich.edu>
gilgalad@caen.engin.umich.edu (Ralph Seguin) writes:
>Howdy.  Considering the high price of the SPEC suite, I was wondering if
>anybody would be willing to get together some sort of PD suite of benchmarks.
>I am thinking (NO! Really? 8-) along these lines:
>
>-a ray trace (DKBTrace gets my vote here!)
>-Large composite number factoring.  Say factor a 20 digit composite composed
> of 2 10 digit primes.
>-Matrix stuff (say inversion, rref, etc).
>-FFT
>-Fractal (say a 1 megapixel render at 1000 iterations of the whole set)
>-some of the more generic, useless benchmarks to please the mindless hordes
>	-dhrystone,whetstone,savage,sieve,...
>-quite a few others which I have been too stupid to mention
>
>Suggestions welcome, and wanted.

Some points:
There are many "public domain" benchmarks: The Livermore Loops,
The Whet and Dhrystones (etc.stones, rolling stones...), our own NAS
kernels are part of SPEC.  I can post those, note it has an FFT and matrix
stuff.  

But....what's representative?  You can't expect to just run a code and get
something meaningful.  Various factors affect the performance of codes.
The NAS kernels take took an 7 hours to run on a VAX 11/780.
1 minute on a Cray X-MP, 27 seconds on a Y-MP.  The matrix multiply
is a 4-way unrolled loop which also requires 64-bit single precision.
The FFT requires 64-bit COMPLEX for a 1K 2-D FFT (128-bit precision)
(do you know how many 32-bit compilers do NOT support a double precision
complex data type? [most]).  Can't run this test.

The fractal and ray trace stuff also requires some consistency.
We hashed this in the TIGPE group of the local SIGGRAPH (what's a
representative scene?).  So prior agreement is required, and then you also 
need to have a base of results.

Not easy.  Have to leave now, got to go benchmarking.  Got a new machine
to try out......

--e.n. miya, NASA Ames Research Center, eugene@eos.arc.nasa.gov
  {uunet,mailrus,most gateways}!ames!eugene
  AMERICA: CHANGE IT OR LOSE IT.