Path: utzoo!utgpu!news-server.csri.toronto.edu!cs.utexas.edu!uunet!timbuk!juniper09!ds
From: ds@juniper09.cray.com (David Sielaff)
Newsgroups: comp.sys.super
Subject: Re: Cray vs. PC Benchmark
Message-ID: <161512.23931@timbuk.cray.com>
Date: 12 Oct 90 21:30:23 GMT
References: <1990Oct9.192119.4453@murdoch.acc.Virginia.EDU>
Organization: Cray Research, Inc., Eagan, MN
Lines: 61

In article <1990Oct9.192119.4453@murdoch.acc.Virginia.EDU> clc5q@shamash.cs.Virginia.EDU (Clark L. Coleman) writes:
>The September 17, 1990 issue of PC WEEK has an advertisement on pages 56-57
>from Everex. The ad claims the following Dhrystone benchmarks:
>
> Everex STEP 486/33  :  34,000 Dhrystones (19.4 MIPS)
> CRAY-X-MP/48        :  17,857 Dhrystones
> IBM 3083            :  16,666 Dhrystones
>
>Is the explanation :
>
> 1) The Everex compiler got rid of a worthless inner loop in a stupid
>     benchmark.
>
> 2) The i486-based PC is really faster on this scalar benchmark than
>     a Cray X-MP/48.
>
> 3) They figured out some way to hamper the Cray for the benchmark.
>
> 4) They're lying.
>
> 5) None of the above.
>
>I realize that Dhrystones are not meaningful for supercomputer applications,
>but I thought that the scalar integer MIPS of a Cray would still exceed a
>PC. Is there any independent source of Cray performance on this test?

I can't claim to be an independent source (I do have a bit of bias in this
particular case ;-) ), but I have a comment or two about the Cray numbers.
Looking at records I found containing benchmarks of various Cray machine/
Cray C compiler combinations over the years.  The most recent benchmark
I found is 27042 Dhrystones on a Cray X-MP/48 with C 4.0 (This compiler was
released about two years ago).  I just ran it myself with our most recent
C compiler on a Cray X-MPEA/464 (most letters and numbers, but essentially
the same machine as far as this benchmark goes), and got 28079 dhrystones.
The benchmark they are quoting is with C 2.0 (about four years old), on
an older generation of X-MP machines with a slower clock speed (9.5 ns
as opposed to 8.5 ns).  Plus the fact that the /4 part of the machine
type means that the machine has four CPU's, which is meaningless for this
benchmark.  All of the timings are for one processor.  Plus we haven't
been making X-MP's for a couple of years (our current product, the Y-MP
has a clock speed of 6 ns, a bit faster).  And finally, Cray's are designed
for floating point, not integer arithmetic, so a lot of extra conversions
from integer format to floating point format (and back) will need to be
done.

If you've stayed with me for this long, what it comes down to is this:
benchmarks, like statistics, say exactly what you want them to.  They are
not lying.  They're just not telling the whole truth.

I hope this didn't sound like a whole bunch of marketing drivel, I just
wanted to point out ways to make benchmarks look exactly how you want
them to, not just on a Cray, but any machine.

>-----------------------------------------------------------------------------
>"We cannot talk of freedom unless we have private property." -- Gavriil Popov,
>Mayor of Moscow, September 11, 1990. |||  clc5q@virginia.edu

Dave Sielaff
Cray Research, Inc.
ds@cray.com
uunet!cray!ds